Method · Background

The Münchhausen trilemma in policy evaluation

Whoever evaluates a measure has to weigh consequences against each other: life years against travel time, tonnes of CO₂ against euros, liberty against safety. There is no final proof for this weighing. Every justification ends somewhere, and where it ends, a setting stands. That is not the weakness of one particular method but a property of every justification. The only question is whether the setting is visible.

The trilemma

Three ends of every justification

In 1968 the philosopher Hans Albert described in his “Treatise on Critical Reason” what happens when a justification is followed to the ground. Every statement needs a justification, and the justification is itself a statement. Exactly three possibilities follow:

  1. Infinite regress. Every justification is justified in turn, without end. The chain never stops and therefore carries nothing.
  2. The circle. The chain returns to a statement that was itself to be justified. The justification rests on itself.
  3. The stop. The chain ends at a statement that is not justified any further. It is set.

The figure in the name is the baron who pulls himself out of the swamp by his own hair: a justification cannot lift itself out of nothing. The ancient form comes from Agrippa, handed down by Sextus Empiricus; of his five modes, regress, circle and assumption form the trilemma. All three ways share one thing: none delivers a final proof. In practice the third remains, the stop. What matters is where the stop is placed and whether anyone sees it.

Application

The stop in every evaluation

A motorway speed limit, according to the available studies, saves life years and tonnes of CO₂ and costs millions of hours of travel time. Whether the measure is better on balance depends on how a life year counts against an hour. No study can answer that question; it can count life years and hours, but it cannot weigh them. At this point every evaluation stops and sets a ratio.

That holds for every form of evaluation. An editorial that recommends a measure sets the ratio in its tone. A cost-benefit report sets it in a number, such as the value of a statistical life, that sits in a footnote. A party position sets it in the programme. A gut feeling sets it without knowing. The difference between these forms is not whether a setting is made, but how visible it is and whether it stays the same at the next topic.

The invisible setting has one property that makes it dangerous: it can drift. Whoever never states the ratio of life year to hour can apply it differently to the speed limit than to the hospital reform, and nobody notices, often not even the person themselves. The wanted result then finds its own yardstick.

The objection

Who judges the judges

The obvious objection to any evaluating body is that it needs a check of its own. The trilemma shows why this objection does not reach its goal: the checking body would have to weigh with values of its own, would therefore need a check in turn, and so on. That is the regress. A council of the wise, an ethics committee, a panel of experts move the stop up one step; they do not remove it.

There is another answer. Instead of handing the setting to a higher authority, it is handed to everyone: it is published, with number and reasoning, and it applies to every evaluation alike. Then every reader can check the setting, reject it, replace it with their own and see what changes as a result. The stop remains, but it is no longer a secret. The verdict belongs to the person reading, not to the editors.

Argupedia

Publishing the setting

Argupedia draws three rules from this. First: every setting is in the Baseline (in German), with derivation, range and change history. A life year counts 40,000 euro-equivalent there, an hour of lifetime 6.85, a tonne of CO₂ 100. Second: every argument names in the value catalogue which value it prices, and the values carry a weight from a fixed register. Third: the same numbers apply to every debate. The yardstick does not drift; when it changes, it changes centrally, dated and for the whole stock.

The settings have not become true by this. They have become checkable. Whoever weighs the life year higher or the hour lower recomputes in the Werte-Shift (in German) and sees which verdicts flip and which hold. Whoever holds a number to be wrong can say which one. And whoever wants to check the whole stock downloads it under Data.

The word for this approach is not objectivity. Objective would be an evaluation without a setting, and there is no such thing. The word is consistency: the same published yardstick for every debate, recomputable by anyone. The trilemma allows no more, and less is deception.

Further

Going deeper