Measurement Over Models

Most blog readers know I strongly prefer measurement over models. In digital security, I think too many practitioners prefer to substitute their own opinions for data, i.e., "defense by belief" instead of "defense by fact." I found an example of a conflict between the two mindsets in Test flights raise hope for European air traffic:

Dutch airline KLM said inspection of an airliner after a test flight showed no damage to engines or evidence of dangerous ash concentrations. Germany's Lufthansa also reported problem-free test flights...

"We hung up filters in the engines to filter the air. We checked whether there was ash in them and all looked good," said a KLM spokeswoman. "We've also checked whether there was deposit on the plane, such as the wings. Yesterday's plane was all well..."

German airline Air Berlin was quoted as expressing irritation at the way the shutdown was decided.

"We are amazed that the results of the test flights done by Lufthansa and Air Berlin have not had any bearing on the decision-making of the air safety authorities," Chief Executive Joachim Hunold told the mass circulation Bild am Sonntag paper.

"The closure of the air space happened purely because of the data of a computer simulation at the Volcanic Ash Advisory Center in London."

I understand that safety officials need to make decisions based on the best information available at the time the decision needs to be made. However, when that information changes, the decision maker should re-evaluate his or her position. This reminds me of the silly policies mandated by various rule-makers regarding password complexity and frequency of change. They are basically completely disconnected with the modern attack and exploitation environment. That thinking recalls a time when guessing credentials or brute-forcing passwords took weeks instead of near-real-time, and was the prevalent way to compromise a system.

Returning to the volcano cloud -- I'm sure safety officials think they are acting in the best interests of passengers, but I don't see the airlines about to take actions that jeopardize their customers. Furthermore, customers who would be wary about flying through or near the ash cloud could decide not to do so. The problem is that safety officials bear none of the cost of their decisions while airlines and customers do.