New “risk intelligence test” is a disappointment

Imagine my surprise, last week, when I read a report announcing that researchers in Germany had created “the first quick test for establishing an individual’s risk intelligence.” After all, I created an online risk intelligence test back in 2009, and I was simply following in the tracks of many researchers before me. What was so special about the new test from Germany, I wondered.

The answer, as I soon found out, is … nothing! In fact the test doesn’t measure risk intelligence at all. To be fair, the authors of the test do not pretend that it does. They claim, instead, that the test measures what they call “risk literacy.” It seems the journalists used poetic license when they described it as a risk intelligence test.

The test is certainly quick. There are only four questions, and it takes only a couple of minutes to answer them. But the questions are the same old probability puzzles that have been the mainstay of books about risk for decades.

For example, one of the questions is as follows:

Out of 1,000 people in a small town 500 are members of a choir. Out of these 500 members in a choir 100 are men. Out of the 500 inhabitants that are not in a choir 300 are men.

 What is the probability that a randomly drawn man is a member of the choir?

Many books that purport to help people think more clearly about risk focus on such analytical puzzles. But although these puzzles can be fun to explore and their solutions are often pleasingly counterintuitive, mastering probability theory is neither necessary nor sufficient for risk intelligence. We know it is not necessary because there are people who have very high risk intelligence yet have never been acquainted with the probability calculus. And mastering probability theory is not sufficient for risk intelligence either, as is demonstrated by the existence of nerds who can crunch numbers effortlessly yet show no flair for estimating probabilities or for judging the reliability of their predictions.

Risk intelligence is not about solving probability puzzles; it is about how to make decisions when your knowledge is uncertain. Outside of some highly structured risk-taking activities, such as are found in casinos and financial markets, dealing with uncertainty is a more useful skill than probability analysis. It is also much easier to learn, primarily because it depends on common sense and simple logic rather than abstract mathematics.

It is depressing to find people still confusing these two things. The journalists who have been waxing lyrical about the German test would do well to read Agatha Christie. Fifty years ago, in The Mirror Crack’d from Side to Side, she showed her disdain for the kind of probability puzzles that the German test regurgitates, when she has Dr Haydock complain: “I can see looming ahead one of those terrible exercises in probability where six men have white hats and six men have black hats and you have to work it out by mathematics how likely it is that the hats will get mixed up and in what proportion. If you start thinking about things like that, you would go round the bend. Let me assure you of that!