Two Axes of LLM Abstention: Answer Correctness and Question Answerability
arXiv:2607.08456v1 Announce Type: new Abstract: A model should refuse two different things: answers it would get wrong, and questions it should not answer at all, such as unanswerable ones or ones…
Read the full articleYou might also wanna read
AI Models Frequently Change Answers When Questioned: The "Are You Sure?" Problem
Ask your AI 'are you sure?' and watch it flip. Models fold 60% of the time because we trained them to please, not push back. The fix isn't b
randalolson.com·3mo ago
Research Reveals LLM Refusal Behavior Is Controlled by a Single Direction in Model Activations
Conversational large language models are fine-tuned for both instruction-following and safety, resulting in models that obey benign requests
arXiv.org·2mo ago
New benchmark reveals AI models often cite wrong sources even when answers are correct
Leading AI models like GPT and Gemini routinely cite text passages in document analyses that don't actually support their answers. Even when
the-decoder.com·1mo ago
Systematic evaluation of 21 LLM-as-a-Judge models reveals reliability flaws and position bias across 541,000 judgments
LLM-as-a-Judge has become the dominant evaluation paradigm for language models, but judge validation in practice relies on exact-match agree
arxiv.org·17d agoSystematic evaluation of 21 LLM-as-a-Judge models reveals reliability flaws and position bias across 541,000 judgments
LLM-as-a-Judge has become the dominant evaluation paradigm for language models, but judge validation in practice relies on exact-match agree
arxiv.org·17d ago
BINEVAL: A Binary Question Framework for Interpretable LLM Evaluation and Self-Improvement
Evaluating LLM outputs remains a major bottleneck in NLP: human evaluation is expensive and slow, lexical metrics correlate poorly with huma
arxiv.org·12d agoBINEVAL: A Binary Question Framework for Interpretable LLM Evaluation and Self-Improvement
Evaluating LLM outputs remains a major bottleneck in NLP: human evaluation is expensive and slow, lexical metrics correlate poorly with huma
arxiv.org·12d ago
Benchmarks in Leipzig: LLMs Solve 98 of 100 Research-Level Math Problems
Between April 1 and May 15, 2026, a group of 49 mathematicians compiled a dataset of research-level mathematics questions with known answers
arxiv.org·1mo ago
Comments
Sign in to join the conversation.
No comments yet. Be the first.