Over half of text-to-SQL benchmark answers are wrong, study finds
A UIUC research team audited leading text-to-SQL benchmarks and found that 52.8% of BIRD Mini-Dev annotations and 62.8% of Spider 2.0-Snow annotations contained incorrect answer keys, as confirmed by…
Read the full articleYou might also wanna read
MCP and Text-to-SQL: How AI Agents Turn Natural Language into Database Queries
A comprehensive guide to text-to-SQL via MCP — covering DBHub, XiYan-SQL, QueryWeaver, Google MCP Toolbox, Oracle AI Database, Wren AI, arch
SQuaD-SQL: Efficient Text-to-SQL with Small Language Models via LLM-Guided Knowledge Distillation
arXiv:2607.08161v1 Announce Type: new Abstract: Text-to-SQL is a fundamental task in natural language processing that enables users to inter
New benchmark reveals AI models often cite wrong sources even when answers are correct
Leading AI models like GPT and Gemini routinely cite text passages in document analyses that don't actually support their answers. Even when
the-decoder.com·1mo ago
Uncertainty in AI-Generated SQL: Why Humans Still Have the Edge
AI-generated SQL queries are getting better, but human judgment in verification still reigns supreme. Why? It boils down to reasoning and sc
Why Semantic Layers Make Enterprise Text-to-SQL Safer
Text-to-SQL generated serious excitement when early demonstrations showed AI assistants turning plain English into working SQL. It also gene
Systematic evaluation of 21 LLM-as-a-Judge models reveals reliability flaws and position bias across 541,000 judgments
LLM-as-a-Judge has become the dominant evaluation paradigm for language models, but judge validation in practice relies on exact-match agree
arxiv.org·17d agoSystematic evaluation of 21 LLM-as-a-Judge models reveals reliability flaws and position bias across 541,000 judgments
LLM-as-a-Judge has become the dominant evaluation paradigm for language models, but judge validation in practice relies on exact-match agree
arxiv.org·17d ago
Comments
Sign in to join the conversation.
No comments yet. Be the first.