#uncertainty-calibration

[ follow ]
#evaluation-metrics #ai-transparency #rlhf #attribution #llm-hallucinations #pre-training-errors
Artificial intelligence
fromMedium
3 hours ago

The case for the uncertain AI: Why chatbots should say "I'm not sure"

Chatbots should explicitly acknowledge uncertainty, cite evidence, and communicate limitations to avoid confidently presenting unverified or incorrect information.
Artificial intelligence
fromInfoQ
3 months ago

OpenAI Study Investigates the Causes of LLM Hallucinations and Potential Solutions

LLM hallucinations largely result from pretraining exposure and evaluation metrics that reward guessing; penalizing confident errors and rewarding uncertainty can reduce hallucinations.
fromZDNET
4 months ago

OpenAI's fix for hallucinations is simpler than you think

"Language models are optimized to be good test-takers, and guessing when uncertain improves test performance," the authors write in the paper. The current evaluation paradigm essentially uses a simple, binary grading metric, rewarding them for accurate responses and penalizing them for inaccurate ones. According to this method, admitting ignorance is judged as an inaccurate response, which pushes models toward generating what OpenAI describes as "overconfident, plausible falsehoods" -- hallucination, in other words.
Artificial intelligence
Artificial intelligence
fromBusiness Insider
4 months ago

Why AI chatbots hallucinate, according to OpenAI researchers

Large language models hallucinate because training and evaluation reward guessing over admitting uncertainty; redesigning evaluation metrics can reduce hallucinations.
[ Load more ]