The Dunning-Kruger effect

He who knows, and knows that he knows, is wise. Follow him.

He who knows, and knows not that he knows, is asleep. Wake him.

He who knows not, and knows that he knows not, is ignorant. Teach him.

He who knows not, and knows not that he knows not, is a fool. Shun him.

This anonymous text is well known. It is generally presented as an Arab or Persian proverb, sometimes as a Chinese proverb, and is even mistakenly attributed to Confucius, as what is written in Analects 17:3 is different. The Dunning-Kruger effect, which refers to a study published in 1999 by these two authors in a journal of the American Psychological Association, could be considered as an experimental study on the first and last lines of the proverb.

To identify the effect that bears their name, Dunning and Kruger conducted and analyzed, with psychology students, a set of tests related to intellectual and social activities in fields such as humor, grammar and logic. They then asked the participants to self-evaluate, by answering the following three questions:

1. What’s the percentage of your ability to respond to these tests, compared to the group of students who have participated in them? If you think you are the worst, answer 0; if you think you are the best, answer 100; if you think you are average, answer 50; and so on. This question detects what students think about their general capacity, in relation to that of the others, to answer a test in this field, rather than the specific case of this test.
2. What’s the percentage of your performance when answering this concrete test, compared to the group of students who have participated? If you think you have got the worst result, answer 0; if you think you have been the best, answer 100; if you think you have done an average test, answer 50; and so on. This question detects what students think about their performance in the specific case of the test they have performed.
3. How many correct answers do you think you got?

The result was spectacular, as shown in the attached figure. Based on their test results, the students were divided into four groups: those with the worst results; two intermediate groups; and the best.

• Students did not differ much in estimating their own ability relative to that of their peers, always considering it above average.
• In general, all the students thought that their performance in the test had been somewhat less than their capacity, albeit narrowly.
• Students who scored worst on the test overestimated their own ability and the results they had obtained. Those in the second group also over-estimated, but not as much. Those in the third rated their results quite accurately. The best, however, significantly underestimated their own performance, or to put it better, overestimated the ability of others, which automatically reduced their own position in the ranking.

As a second part of the experiment, each participant in the two groups of the worst and the best was given five tests, filled out by as many colleagues, to be analyzed, rated and compared with their own, which gave them some information about the capacity of the other participants. They were then asked to re-estimate their own performance, by answering again the three questions given above.

The worst-performing students kept their estimates essentially identical. The analysis of the work of their colleagues did not help them rate themselves better. Despite receiving additional information, the incompetent were unable to change their idea about ​​themselves, because they couldn’t detect that others had done better. In contrast, the most competent significantly improved their own estimates, which shows that they had discovered that many of their classmates had solved the test significantly worse than they had.

These conclusions should be applied in practical life, when so-called experts participate in debates, and even give lectures, while in fact they are incompetent in the subjects they are speaking about. The studies by Dunning and Kruger show that they are not just incompetent, but are probably unable to recognize their incompetence. According to the proverb quoted at the beginning, they should be described as fools and be shunned as much as possible.

The same post in Spanish
Thematic thread on Natural and Artificial Intelligence: Preceding Next

Manuel Alfonseca