AI Advice Makes People Three Times Less Accurate, Twice as Confident

A five-experiment study finds that access to AI advice collapsed people's willingness to say "I don't know" from 44% to 3%, while accuracy dropped from 27% to 9% — yet confidence nearly tripled.

A new preprint from researchers at École Normale Supérieure and Sapienza University of Rome delivers a stark warning about how AI is reshaping human judgment. Across five experiments with 3,132 participants, merely having access to AI advice nearly eliminated people's willingness to admit ignorance — even when that advice was systematically wrong.

The numbers are dramatic. Without AI, 44% of participants said "I don't know" when faced with difficult questions. With AI access, that dropped to 3%. Accuracy fell from 27% to 9%, but confidence soared from 30% to 76%. People became three times less accurate yet more than twice as sure of themselves.

The researchers deliberately used questions where AI models typically fail — visual details from films, such as the colour of a team's uniform in Bend It Like Beckham. The AI gave fluent-sounding but incorrect answers, and participants followed them off a cliff.

Monetary incentives for accuracy barely moved the needle. Willingness to admit ignorance rose only to 8%, and accuracy improved to just 16% — both still far below the no-AI baseline. The study's lead author, Valerio Capraro, emphasized that "the capacity to say 'I don't know' is very important because it represents the recognition of the limits of our own knowledge."

The findings echo what Wharton researchers earlier this year called "cognitive surrender" — the phenomenon of people accepting AI outputs without scrutiny. Capraro expressed particular concern about children growing up with these systems before developing their own critical thinking skills.

Sources: The Next Web, OSF Preprints