New benchmark reveals AI models often cite wrong sources even when answers are correct

Summary

Researchers at Peking University have developed CiteVQA, a new benchmark that tests whether AI models can correctly cite source documents when answering questions. The study reveals that leading AI models like GPT and Gemini frequently suffer from "attribution hallucination" — providing correct answers but pointing to wrong or irrelevant source passages. This poses significant risks for regulated fields like law, medicine, and financial auditing, where traceability and evidence verification are critical. CiteVQA is the first systematic benchmark designed to evaluate both answer accuracy and citation correctness in document analysis tasks.