Scenario planning and probabilities

First, a caveat; I don’t know much about scenario planning, so the following comments may come across as rather simplistic to those well versed in this area.  Also, it is probably rather presumptuous to be so critical of something I know so little about. So consider this post as an opening gambit rather than a considered conclusion.

I recently exchanged a few emails with a guy who does scenario planning for a non-profit organization.  When I asked him if he got people to attach numerical probabilities to each scenario, he replied: “We don’t do probabilities, but instead run workshops and interviews to get a sense of where people’s mental models are in terms of how things might turn out.” The problem with this is that weasel word, “might,” which could mean anything from “extremely unlikely” to “almost certain”.

For example, suppose the folks at the Pentagon are mapping out possible scenarios that might follow a US invasion of Syria, such as:

  • Invasion is successful with minimal human and financial cost, Syrians welcome the troops and quickly set up a prosperous, democratic and liberal society that becomes a strong US ally and force for positive change in the Islamic world.
  • Invasion is a complete disaster with massive cost and casualties, resulting in a devastated Syria split into violent fiefdoms, including one run by Assad and another by Al Qaeda. The US is humiliated both militarily and by revelation of major scandals and atrocities. Many US troops are prisoners.

Plus various intermediate possibilities.

So far, so good.  The precise details in each scenario are not that important, since the scenarios are really just placeholders for a set of outcomes arranged in order of preference.  The real problems begin when we go from scenarios to decisions. For unless we have some idea of how likely each scenario is, it will be impossible to assess the expected utility of various mitigating strategies.  There may be no point in spending billions of dollars to avert a worst-case scenario if the probability of that scenario occurring is very low.

The Intergovernmental Panel on Climate Change (IPCC) attaches numerical probabilities to various scenarios it discusses in its reports. Everyone else who does scenario planning should do the same.

4 thoughts on “Scenario planning and probabilities

  1. Making predictions is a messy business, albeit it both necessary and important. I think one can dramatically improve the predictions industry is there is more post-fact review and introspection to understand the failings of past predictions, and to understand why these predictions failed as well. Otherwise, whether we use numbers or not, the prediction industry will continue to happily look forward, never checking the rear-view mirror, and never really tuning their capacity to make better predictions in the future.

  2. So what numeric probability did the rigorously scientific IPCC attach to the Himalayan glacier melt? Surely you could use a more credible scientific body as a standard, or are you simply doing your ‘duty’ as a change agent?

    • The fact that the IPCC made such a stupid mistake regarding the Himalayan glacier melt is entirely irrelevant to the point I was making in this piece. In assigning numerical probabilities to the various possible scenarios it envisages, the IPCC is setting a good example. Its peer review process is unfortunately much less worthy of emulation.

  3. Yes, probabilities are useful, but scenarios can be useful without them. If your decision is merely invade/don’t invade, then looking at a list of scenarios is only mildly helpful. There might be a plausible one that is intolerable, so you don’t invade regardless of the potential upside. Or the good ones might so outweigh the bad ones that you think invasion is a good idea for any non-extreme assignments of probabilities.

    But the real value of scenarios without probabilities is for detailed decisions. Looking at the two above, for example, you might think about whether to adhere to the Geneva Convention if you invade. In the good case, it would help a little by increasing support for something that might reduce the harm of war. In the bad case it helps enormously as it protects your prisoners, makes peace easier to negotiate and reduces the probability that you will be convicted of war crimes. Scenarios can make some detail decisions obvious.

Leave a Reply

Your email address will not be published. Required fields are marked *