Consequences of statistical ignorance

In a study performed a few years ago [1], the following problem was proposed to 18 expert consultants for AIDS patients:

Helen has tested positive for AIDS. How likely do you think she is really an AIDS patient? What would you advise her?

The input data are:

1.      The probability of having AIDS, when one belongs to a population without special risk, is 1 in 10,000.

2.      The sensitivity of the AIDS test is 99.9%. In other words, the probability of a false negative is 0.1%.

3.      The specificity of the AIDS test is 99.99%. In other words, the probability of a false positive is 0.01%.

The result of the study was as follows:

·         The 18 experts agreed that the probability that Helen is an AIDS patient is greater than 90%. Most thought that the probability is greater than 99%. Some even claimed that is greater than 99.9%.

·         All experts said that they would advise Helen to inform her family, make everybody test for AIDS, and start taking medication.

To find what really happens, we’ll consider a random population of 10,000 people without special AIDS risk and we’ll test them all. What are the results?

1.      As the probability of having AIDS in this population is 1 in 10,000, there will probably be a person with AIDS and 9,999 AIDS-free:

2.      Since the probability of a false negative is 0.1%, it is almost certain that the person with AIDS will test positive.

3.      Since the probability of a false positive is 0.01% (1 in 10,000), when testing the 9,999 individuals who do not have AIDS, one of them will test positive without having AIDS (the probability of this is 0.9999).

4.      Summarizing: when testing those 10,000 people for AIDS, two will test positive. One is an AIDS patient, the other is not.

5.      Therefore, the probability that Helen is really an AIDS patient is 50% (the same as getting heads when flipping a coin).

6.      The advice the experts should have given Helen is: repeat the test in another laboratory.

Are the experts really experts? In whose hands are we?

[1] E.Kurz-Milcke, G.Gigerenzer, L.Martignon, “Transparency in risk communication: graphical and analog tools,” in Strategies for risk communication: evolution, evidence, experience, Annals of the New York Academy of Sciences, vol.1128, 2008.

The same post in Spanish
Thematic Thread on Statistics: Next
Thematic Thread on Linguistics and Medicine: Next

Manuel Alfonseca