AI models sometimes guess when unsure, producing plausible but false statements that undermine trust. AI that reasons reliably and admits when it does not know would be more useful and more trustworthy for the people who rely on it.
What we know
Reasoning models have improved sharply on mathematics and code.
What blocks us
They still state false things confidently and misjudge their own uncertainty.
Current leads
Automatic verification of answers, training on checked results, calibrated confidence.
Sub-questions
How can an AI learn to recognize the limits of its own knowledge?
Would AIs make up fewer false answers if tests stopped rewarding guessing over saying 'I don't know'?
Why can extra training make a model worse at judging how confident it should be?