The correction nobody read

In 1999 David Dunning and Justin Kruger gave people a test, asked them to estimate how well they had done, and reported that the people who scored worst overestimated themselves the most.

You know the rest, because the finding escaped. It is now a load-bearing beam of internet argument. A way of saying you are too stupid to know you are stupid while sounding as though you are citing something.

There is a substantial literature arguing the effect is much weaker than that, and possibly an artefact of how the original analysis was done. Edward Nuhfer and colleagues published detailed critiques in 2016 and 2017. Gilles Gignac and Marcin Zajenkowski published another in 2020. The critiques collectively have on the order of ninety times fewer citations than the paper they are critiquing.

That ratio is the thing I want to talk about. Not who is right.

The statistical objection, briefly

The core complaint is about what gets plotted against what.

The classic chart shows self-assessment error. Your estimate minus your actual score, on one axis, and your actual score on the other. Both axes contain your actual score. Plot any two variables where one is built out of the other and you will get a downward slope whether or not anything psychological is happening, because low scorers can only err upward and high scorers can only err downward. There is nowhere else for the points to go.

Critics argue that a good deal of the famous pattern falls out of that arrangement alone. Defenders argue the effect survives better analyses in weakened form. I am not equipped to settle it, and this post is not trying to.

What is not in dispute is that a serious, technical, decade-old challenge exists, and that almost nobody who invokes the effect has encountered it.

Corrections travel worse than findings

Here is what I think is actually going on, and it is not that people are lazy.

A finding and its correction are not the same kind of object. The finding is a story: it has characters, a shape, a punchline, and it explains something you already suspected about your colleagues. The correction is a methodological argument about autocorrelation. One of these survives being retold at a dinner table. The other does not survive being retold at all. It can only be read.

So the two propagate through completely different channels at completely different speeds. The finding moves by anecdote, which is fast and lossy. The correction moves by citation, which is slow and requires you to already be in the literature. By the time a correction is published, the finding has usually left the building and is out in the general culture where citations cannot reach it.

This is why “it was later disputed” so rarely catches up with anything. The dispute is not competing with the original paper. It is competing with a folk memory of the original paper, and folk memory does not read journals.

The uncomfortable part

I have quoted Dunning-Kruger in talks. More than once. I did not check it, because it did not feel like the sort of thing that needed checking. It felt like background knowledge, the kind of fact you can lean on without a citation because everyone already agrees.

That feeling is the actual finding here, and it is worth more than the effect it concerns.

The claims most likely to be wrong in your head are the ones that arrived without a source, because they arrived so long ago and from so many directions that they stopped feeling like claims at all. They do not present as things you believe. They present as things you know.

A useful exercise, and an unpleasant one: pick three things you say confidently in your field that you have never personally traced to a source. Not things you are unsure about. You already handle those carefully. Things you are certain about and have never checked.

That is where your Dunning-Kruger is, whatever the state of the literature turns out to be.