Scenario planning and probabilities

First, a caveat; I don’t know much about scenario planning, so the following comments may come across as rather simplistic to those well versed in this area.  Also, it is probably rather presumptuous to be so critical of something I know so little about. So consider this post as an opening gambit rather than a considered conclusion.

I recently exchanged a few emails with a guy who does scenario planning for a non-profit organization.  When I asked him if he got people to attach numerical probabilities to each scenario, he replied: “We don’t do probabilities, but instead run workshops and interviews to get a sense of where people’s mental models are in terms of how things might turn out.” The problem with this is that weasel word, “might,” which could mean anything from “extremely unlikely” to “almost certain”.

For example, suppose the folks at the Pentagon are mapping out possible scenarios that might follow a US invasion of Syria, such as:

  • Invasion is successful with minimal human and financial cost, Syrians welcome the troops and quickly set up a prosperous, democratic and liberal society that becomes a strong US ally and force for positive change in the Islamic world.
  • Invasion is a complete disaster with massive cost and casualties, resulting in a devastated Syria split into violent fiefdoms, including one run by Assad and another by Al Qaeda. The US is humiliated both militarily and by revelation of major scandals and atrocities. Many US troops are prisoners.

Plus various intermediate possibilities.

So far, so good.  The precise details in each scenario are not that important, since the scenarios are really just placeholders for a set of outcomes arranged in order of preference.  The real problems begin when we go from scenarios to decisions. For unless we have some idea of how likely each scenario is, it will be impossible to assess the expected utility of various mitigating strategies.  There may be no point in spending billions of dollars to avert a worst-case scenario if the probability of that scenario occurring is very low.

The Intergovernmental Panel on Climate Change (IPCC) attaches numerical probabilities to various scenarios it discusses in its reports. Everyone else who does scenario planning should do the same.