A launch pad on a frosty morning: the shuttle stands at the launch tower, icicles hang from railings and steelwork in front, a thermometer is fixed to a post. Engraving-style illustration.
Decision under uncertainty

Challenger 1986: a risk nobody could put a number on

A risk without a number is not a risk of zero.

On the evening of 27 January 1986 the engineers of the rocket manufacturer recommended postponing the launch: there was no experience with the cold expected the next morning. Four managers decided otherwise. The decision map shows what had to be weighed and why a missing number is no reason to launch.

29 °F
calculated O-ring temperature at launch (−2 °C). The coldest launch before that was at 53 °F (12 °C).
Rogers Commission, hearing of 25 February 1986
4 of 4
flights below 65 °F (18 °C) had thermal distress on the O-rings. Above that it was 3 of 20.
Rogers Commission, Volume 1, Chapter VI
975 days
the shuttles stayed on the ground after the accident, until 29 September 1988.
NASA, missions STS-51L and STS-26

A known problem and a cold morning

The shuttle's solid rocket boosters are built from segments. In the joints between them, two rubber rings hold back the hot gases. From the second flight in 1981 onward, technicians kept finding rings on recovered boosters that were scorched or partly burned away [2]. In December 1982 NASA put the joint in its highest criticality class: if it fails, mission, vehicle and crew are lost [2].

The sources use degrees Fahrenheit: 32 °F is the freezing point, 53 °F is 12 °C. The coldest launch up to then was the one in January 1985, with 53 °F at the rings. It showed the worst damage: hot gas had passed the first ring at two joints, and for the first time heat had reached the second ring [2]. In a bench test a ring lost contact with the sealing surface for 2.4 seconds at 75 °F (24 °C). At 50 °F (10 °C) it did not regain contact within ten minutes [2].

On 31 July 1985 the engineer Roger Boisjoly warned his company Morton Thiokol, the maker of the boosters, in writing: the result could be a catastrophe with loss of human life [2]. Frost was forecast in Florida for the morning of 28 January 1986. From the forecast Thiokol calculated 29 °F (−2 °C) for the rings. That is 24 °F, or 13 degrees Celsius, less than on the coldest flight [5]. The decision that evening was made with this number, so the map uses it as well. By then the launch had been postponed three times and scrubbed once [7].

The night before the launch

On the evening of 27 January, 34 specialists from Thiokol and NASA joined a call from three sites. Thiokol's recommendation was on a handwritten chart: the rings must be at least 53 °F at launch [1]. NASA managers objected sharply. Thiokol asked for five minutes for an internal caucus. It lasted half an hour [1].

In that caucus the most senior manager asked the head of engineering to take off his engineering hat and put on his management hat. Four managers decided; the engineers were no longer asked [1]. At about 11:45 p.m. Thiokol signed the recommendation to launch [1].

Challenger lifted off on 28 January at 11:38 a.m. By then the air had warmed to 36 °F (2 °C) [6]. After the night of frost the boosters were colder than the air: for the spot where the joint failed, the Commission estimated 28 °F, give or take 5 °F [13]. Thiokol's calculation of the evening before was therefore accurate. A joint on the right booster failed. After 73 seconds the shuttle broke apart. All seven people on board died [6].

What was said that evening

The investigating commission asked for the sentences of that evening under oath. That the risk could not be put into a number made the evening hard, and that is normal for such a question. But the gap was not filled with arguments. Of the seven sentences on record, none names an impact of launching. Four apply pressure or change who decides. Three concern the plausibility of the risk, and they are the only thing the evening needed to talk about. Nobody said what launching that morning was worth: according to the testimony of Thiokol manager Jerald Mason, the schedule was not discussed in the internal caucus [5]. According to Boisjoly, none of the engineers spoke in favor of launching [1].

“My God, Thiokol, when do you want me to launch, next April?”

Lawrence Mulloy, NASA, as the sentence was reported. Before the Commission he gave a milder version: with that limit, launches might not be possible until April. [1]

not an argument

The sentence argues against a general temperature limit. The decision was about one launch, and a warmer day would have been enough for it. The sentence does not name what waiting costs.

“I stated I was somewhat appalled”

George Hardy, NASA, about Thiokol's recommendation [1]

not an argument

A reaction that carries rank. It says nothing about value, size or plausibility, and it makes every further objection more costly.

“You guys are generating new Launch Commit Criteria”

Lawrence Mulloy, NASA, as testified by Allan McDonald [1]

not an argument

Whether a rule for temperature exists does not change whether the joint seals at 29 °F. The 24 flights before were all at 53 °F or warmer.

“take off his engineering hat and put on his management hat”

Jerald Mason, Thiokol, to head of engineering Robert Lund (as testified by Roger Boisjoly) [1]

not an argument

The sentence changes who decides. It does not change the evidence.

“Temperature data not conclusive on predicting primary O-ring blow-by”

Thiokol's written launch recommendation [1]

concerns plausibility

Only the flights with damage were compared that evening. With all 24 flights the pattern is there: four of four below 65 °F, three of twenty above. And not conclusive means unknown, not safe.

“If the primary seal does not seat, the secondary seal will seat”

Thiokol's written launch recommendation [1]

concerns plausibility

Since December 1982 the joint had been in the criticality class without a reliable backup. On the coldest flight, heat had already reached the second ring.

“Demonstrated sealing threshold is 3 times greater than 0.038″ erosion experienced on SRM-15”

Thiokol's written launch recommendation [1]

concerns plausibility

The margin applies to erosion of a ring that is seated. The open question was whether a cold, hard ring seats in time. The bench test at 50 °F said it does not.

The question was clear, the method was missing

The question of that evening was clear: whether Challenger could launch despite the cold. It comes down to a single quantity, the probability that a joint fails to seal at 29 °F. That quantity could not be measured. Roger Boisjoly was asked to quantify his concern and said he could not. He only knew that they were moving away from the safe range of the existing data [1].

Not quantifiable does not mean unknown. Four things were established. It was the coldest launch, 24 °F below all experience. There had been thermal distress on the rings several times, the worst on the coldest flight up to then [2]. Cold rubber hardens and follows the joint more slowly, as the bench test had shown [2]. And NASA itself kept the joint in the class in which a failure costs vehicle and crew [2]. Each of these points moves the probability away from zero and above that of an ordinary flight. None of them yields a number; together they yield a direction.

What was missing was a procedure that can work with knowledge of this kind. The analysis of all flights by temperature, which shows the pattern, was not carried out by anyone before the accident. The Commission found that NASA and Thiokol were therefore unprepared to evaluate the risk of launching in conditions more extreme than any before [2]. Nor was there a shared format that evening in which the value, size and plausibility of an objection stand side by side. In such a format the question would have been whether the risk lies above what is acceptable for a flight. That was not discussed.

Where the method is missing, what decides is what people do without one. Whoever cannot prove something loses: the engineers were expected to show that launching was unsafe [1]. Rank settles it: in the end four managers voted [1]. And habit reassures: 24 flights had gone well although the rings kept showing damage. The physicist Richard Feynman, a member of the Commission, called this a kind of Russian roulette in which every flight survived lowers the standard [2]. These are not the failings of individuals. This is how people decide under uncertainty when nothing forces them into a different format.

A standard for weighing asks of every sentence which row of the map it changes. “Next April”, being appalled and the management hat change none and drop out. What remains is a single question: whether the risk at 29 °F lies above what is acceptable for a flight. That question is hard because the situation was hard. The evening could have concentrated on it, and it had an answer even without an exact number.

The Commission did not see the risk only on that evening. In its judgment the damage history presented to NASA Headquarters in August 1985 already required corrective action before the next flight [2]. The cold of 28 January came on top of a risk that was already too large.

The decision as a map

The map puts two futures side by side. In one, Challenger launches on the morning of 28 January with a calculated 29 °F at the rings. In the other, NASA waits until the rings are at least 53 °F: the coldest value the system had ever flown with.

The first state shows how the evening did the sums. Robert Lund later testified that Thiokol had to prove it was not ready to launch [1]. A risk that could not be proven therefore counted as zero. The second state scores the same row with the evidence of that evening.

In this row plausibility does not measure whether the joint fails. Nobody would launch at a probability of 1 in 10; the standard for a safe launch is far stricter. It measures how well supported it is that the launch lies above the risk NASA itself accepts for a flight. The risk itself is in the size of the row.

Decision · Launch of Challenger on 28 January 1986V × I × P ÷ 10

F1: launch in the morning, with a calculated 29 °F at the O-rings.  F0: wait until the rings are at least 53 °F.

How the evening did the sums

Nobody voiced the reasons for launching. The risk counted as unproven and dropped out of the calculation.

Arguments for launching in the morning Side total 10.2
The 1986 flight schedule loses no further days not voiced 5.4

Size a few days, two weeks in the unfavorable case

How it is scored
V 6
A reliable schedule was a declared goal: 15 flights were planned for 1986, 24 a year for 1990. That carries weight, but less than lives and the vehicle.
I 1
The launch slips by a few days, by two weeks in the unfavorable case, and the following flights move back. Nobody is harmed and no hardware is lost. Against the stand-down after a loss (623 days after the Apollo fire in 1967), two weeks are one fortieth, and the stand-down is only part of the loss.
P 9
Certain. Launching in the morning saves the days.
The May launch windows of the Galileo and Ulysses probes stay safe not voiced 1.2

Size a one-year wait for two missions if May is missed

How it is scored
V 6
The same class of value as the schedule: missions that fly as planned.
I 2
If the two probes miss their window in May, two of about 15 flights that year wait a year for the next one. The probes are kept and fly later. That is more than a few days of delay and still a small fraction of the whole program.
P 1
Low. Four months and several flights lie between a few days of delay in January and a missed window in May. Nobody showed that the delay would carry that far.
Thiokol does not oppose its most important customer not voiced 3.6

Size the relationship with the client in a contract year

How it is scored
V 6
A real good for the manufacturer: in January 1986 NASA had announced its intention to establish a second supplier for the rocket motors.
I 2
The client is displeased, and Thiokol enters the year in which NASA is looking for a second supplier with a disagreement on record. No contract is at stake that evening. Measured against the loss of an orbiter, which hits the manufacturer as well, that is small.
P 3
Weakly supported. Nothing shows that a technically reasoned recommendation costs the contract. NASA manager George Hardy said that evening that he would not recommend launching over Thiokol's objections.
Arguments for waiting Side total 0.0
The launch is above the risk that is acceptable for a flight dismissed as unproven 0.0

Size seven lives, one of four orbiters with its payload, the fleet grounded, at a risk of loss of about 1 in 10

How it is scored
V 10
Human lives are at the top of the scale. Mission success and the continuation of the program come on top of that.
I 10
If the joint fails, the seven people on board die. The orbiter is lost, one of four, and with it the payload, including a NASA communications satellite. The remaining orbiters stay on the ground until the cause is found and fixed: after the Apollo fire in 1967 that took 623 days. Added to that is the damage to trust in a program that had planned 15 flights for 1986. NASA itself had kept the joint since December 1982 in the class in which a failure costs mission, vehicle and crew. This full loss is a hundred times as large as two weeks of delay. It enters the map with the risk the cold adds: the calculation uses 1 in 10, that is one tenth of the full loss. That is an assumption; with 1 in 25 waiting is still ahead. The row remains the largest difference on the table, which is why it stands at 10.
P 0
This is how the evening did the sums: the engineers could not prove that the joint would fail at 29 °F. So the row dropped out of the calculation as if the risk had been ruled out.
Reading Launch: lead 10.2 On this map launching wins because the other side is empty. Zero on the scale is an exclusion, though, and an exclusion needs a reason. “Not proven” is none: for 29 °F there was no flight and no test that ruled the risk out.

With the evidence of that evening

The same four rows. The risk now carries the value that the data of that evening support at the least.

Arguments for launching in the morning Side total 10.2
The 1986 flight schedule loses no further days 5.4

Size a few days, two weeks in the unfavorable case

How it is scored
V 6
A reliable schedule was a declared goal: 15 flights were planned for 1986, 24 a year for 1990. That carries weight, but less than lives and the vehicle.
I 1
The launch slips by a few days, by two weeks in the unfavorable case, and the following flights move back. Nobody is harmed and no hardware is lost. Against the stand-down after a loss (623 days after the Apollo fire in 1967), two weeks are one fortieth, and the stand-down is only part of the loss.
P 9
Certain. Launching in the morning saves the days.
The May launch windows of the Galileo and Ulysses probes stay safe 1.2

Size a one-year wait for two missions if May is missed

How it is scored
V 6
The same class of value as the schedule: missions that fly as planned.
I 2
If the two probes miss their window in May, two of about 15 flights that year wait a year for the next one. The probes are kept and fly later. That is more than a few days of delay and still a small fraction of the whole program.
P 1
Low. Four months and several flights lie between a few days of delay in January and a missed window in May. Nobody showed that the delay would carry that far.
Thiokol does not oppose its most important customer 3.6

Size the relationship with the client in a contract year

How it is scored
V 6
A real good for the manufacturer: in January 1986 NASA had announced its intention to establish a second supplier for the rocket motors.
I 2
The client is displeased, and Thiokol enters the year in which NASA is looking for a second supplier with a disagreement on record. No contract is at stake that evening. Measured against the loss of an orbiter, which hits the manufacturer as well, that is small.
P 3
Weakly supported. Nothing shows that a technically reasoned recommendation costs the contract. NASA manager George Hardy said that evening that he would not recommend launching over Thiokol's objections.
Arguments for waiting Side total 50.0
The launch is above the risk that is acceptable for a flight 50.0

Size seven lives, one of four orbiters with its payload, the fleet grounded, at a risk of loss of about 1 in 10

How it is scored
V 10
Human lives are at the top of the scale. Mission success and the continuation of the program come on top of that.
I 10
If the joint fails, the seven people on board die. The orbiter is lost, one of four, and with it the payload, including a NASA communications satellite. The remaining orbiters stay on the ground until the cause is found and fixed: after the Apollo fire in 1967 that took 623 days. Added to that is the damage to trust in a program that had planned 15 flights for 1986. NASA itself had kept the joint since December 1982 in the class in which a failure costs mission, vehicle and crew. This full loss is a hundred times as large as two weeks of delay. It enters the map with the risk the cold adds: the calculation uses 1 in 10, that is one tenth of the full loss. That is an assumption; with 1 in 25 waiting is still ahead. The row remains the largest difference on the table, which is why it stands at 10.
P 5
The question for this row is whether the launch lies above the risk NASA accepts for a flight. Its engineers put the risk of loss on an ordinary flight at about 1 in 100, management at 1 in 100,000. There is a good case that the cold leads beyond that: thermal distress on all four flights below 65 °F, the worst on the coldest; a bench test in which the ring found no contact for ten minutes at 50 °F; a launch 24 °F below all experience. There was no measurement for 29 °F, and part of the data could be read differently. Hence 5: quite possible, not established. Given what is at stake in this row, that is enough.
Reading Wait: lead 39.8 A single row carries the decision. The launch side holds days in the schedule; the other side holds seven people and an orbiter. At that ratio, waiting wins as soon as the cold raises the risk by more than about 1 in 50. The plausibility of that does not have to be high. It is enough that it cannot be ruled out.

Decision map: two states of the same decision. The numbers in the four rows are the same, except for the plausibility of the risk.

How to read the map

Each row names exactly one impact: one difference between the two futures. V is the weight of the good at stake (0 to 10), I is the size of the difference (10 is the largest on the table, all others are in proportion to it), P is the plausibility given the evidence on the day of the decision (0 to 10). The numbers are assumptions of this reconstruction, not measurements. Each one carries its reason so that it can be challenged.

The scoring uses what was known on the day of the decision. What became known later is in the last section and enters no number on the map.

The missing number was no reason to launch

24 flights before the launch · joint temperatureAs of 27 January 1986

Flight with thermal distress on an O-ring Flight without damage

NO FLIGHT IN THIS RANGE 30 °F -1 °C 40 °F 4 °C 50 °F 10 °C 60 °F 16 °C 70 °F 21 °C 80 °F 27 °C JOINT TEMPERATURE AT LAUNCH 28 JANUARY: 29 °F CALCULATED COLDEST LAUNCH SO FAR: 53 °F
Data as a table
FlightJointThermal distress
STS-1 66 °F / 19 °C no
STS-2 70 °F / 21 °C yes
STS-3 80 °F / 27 °C no
STS-4 80 °F / 27 °C no
STS-5 68 °F / 20 °C no
STS-6 67 °F / 19 °C no
STS-7 72 °F / 22 °C no
STS-8 73 °F / 23 °C no
STS-9 70 °F / 21 °C no
41-B 57 °F / 14 °C yes
41-C 63 °F / 17 °C yes
41-D 70 °F / 21 °C yes
41-G 67 °F / 19 °C no
51-A 67 °F / 19 °C no
51-C 53 °F / 12 °C yes
51-D 67 °F / 19 °C no
51-B 75 °F / 24 °C no
51-G 70 °F / 21 °C no
51-F 81 °F / 27 °C no
51-I 76 °F / 24 °C no
51-J 79 °F / 26 °C no
61-A 75 °F / 24 °C yes
61-B 76 °F / 24 °C no
61-C 58 °F / 14 °C yes

Fig. 1: All four flights below 65 °F had thermal distress on the O-rings; above 65 °F it was three of twenty. The launch was 24 °F below the coldest one so far. Data: Rogers Commission 1986, Volume 1, Chapter VI.

Risk of loss · tipping point and estimatesTipping point of the map
LAUNCH LEADS WAIT LEADS Tipping point: 1 in 50 1989 recalculation from the flight data at least 13 in 100 NASA management: risk per flight 1 in 100,000 NASA engineers: risk per flight 1 in 100 Assumption of the map 1 in 10

Fig. 2: Launching only leads if the cold raises the risk of loss by less than 1 in 50. The axis is logarithmic: each step is a factor of ten.

The data in the first figure were on hand on the evening of 27 January. They did not exist as a picture: nobody plotted them in this form before the accident. Only the flights with damage were compared, and in that selection temperature looks unremarkable: there was damage at 53 °F and at 75 °F [2]. The pattern only appears once the flights without damage are included.

Nobody could say how likely a failure at 29 °F was. The map does not need that number. It needs the answer to a simpler question: on which side of the tipping point the truth lies. The tipping point is at about 1 in 50. The added risk from the cold would have had to be that small for the days gained to outweigh it.

For comparison: NASA engineers put the risk of losing vehicle and crew on a normal flight at about 1 in 100, management at 1 in 100,000 [4]. So the cold could at most triple the risk of a normal flight. Against that stood four of four cold flights with damage, a bench test, and 24 °F of distance from all experience.

A statistical recalculation from 1989, using the flight data from before the accident, arrives at a probability of at least 13 in 100 [8]. It works with 31 °F and is thus close to the 29 °F of that evening. That number did not exist that evening. It does not enter the map and only shows where the truth lay: far above the tipping point.

What would have changed the decision

The evening apparently lacked a procedure for weighing a risk without an exact number instead of setting it to zero. With such a procedure, three things change.

First, the burden of proof. Whoever wants to launch at the highest value and the largest impact has to show that the risk is small enough. That evening it was the other way round: Roger Boisjoly testified that the engineers were expected to prove beyond any doubt that launching was not safe [1]. On the map no side wins because the other cannot prove something. Every row needs its own reason.

Second, the empty side. No argument for launching is on record from that evening that names an impact of this one launch. Written down, it would have been days in the schedule, set against seven lives and one of four orbiters. Nobody voiced that comparison, so nobody had to object to it.

Third, a language across ranks. Being appalled and management hats are not entries on a map. An engineer who gives reasons for the plausibility and a manager who sets it lower are talking about the same row. The room sees whose reason holds.

Waiting had a price: more days of delay after four failed dates, in a year with 15 planned flights [3]. The map does not hide it. It shows how small it is next to what was on the other side.

What happened afterwards

The commission chaired by William Rogers found that Thiokol's management had reversed its position at the urging of NASA and against the views of its own engineers, in order to accommodate a major customer [1]. It found no evidence of pressure from outside NASA [3].

The shuttles stayed on the ground for 975 days, until 29 September 1988 [10]. After the Apollo 1 fire the pause had lasted 623 days [11][12]. The physicist Richard Feynman wrote in his appendix to the report that a successful technology must put reality above public relations, because nature cannot be fooled [4].

The same pattern

A risk without a number is not a risk of zero.

A risk that cannot be quantified counts as zero, and those who warn have to provide the proof: the investigation reports on major accidents describe this pattern again and again.

  • 2003

    Columbia

    At launch a piece of insulating foam struck the wing. Engineers asked three times for images of the orbiter in space; program management cancelled the request. The investigation board writes that NASA inverted the burden of proof [14].

  • 2010

    Deepwater Horizon

    A pressure test on the well showed an anomalous reading, and the crew explained it away. According to the national commission's report, important decisions were made without any formal risk analysis. Eleven people died [15].

  • 2011

    Fukushima Daiichi

    According to the Japanese parliament's report, the operator and the regulator knew that flooding would cause a total loss of power. The operator called an assessment based on probabilities technically too uncertain and used that to postpone countermeasures [16].

Ask the bot

The bot explains the method and the debate on this page. It answers only from Argupedia's content and does no research of its own.

Before you ask

  • The answers are written by an AI model (Claude Haiku by Anthropic). They may be wrong or incomplete. The content of the pages is authoritative.
  • The bot gives no legal, financial, medical or voting advice.
  • What you type is sent to Anthropic in the USA. Do not enter names, contact details or other personal information.

The terms of use (section 9a) and the privacy notice for the chat apply.