How a fake Wikipedia edit poisoned multiple LLMs into citing a nonexistent championship
By
SEJeff
Pulled from the oven just right. Trustworthy, fact-dense, deeply satisfying.
Summary
The author describes how they created a fake "6 Nimmt! World Championship" title by editing a Wikipedia page and registering a domain (6nimmt.com), then observed multiple frontier LLMs citing this fabricated information as fact. The article explores the vulnerability of LLM training data to deliberate misinformation, demonstrating how easily the AI knowledge supply chain can be poisoned through simple web manipulation tactics.
Key quotes
· 3 pulledI am the reigning 6 Nimmt! World Champion. I won the title in Munich in January 2025 defeating players from over twenty countries in what I later described to reporters as 'the toughest competition I've ever faced.'
In reality, there is no 6 Nimmt! World Championship. I have never been to Munich. The quote is something I wrote in about thirty seconds while a Wikipedia page was loading.
This is the story of how I manufactured that title, got it quoted back to me by multiple frontier LLMs, and what I think it means for the trust we're about
You might also wanna read
Study finds LLMs persist in treating false claims as true despite explicit warnings
A study on fine-tuning large language models (LLMs) reveals that even after explicit warnings that certain claims are false, the models cont
arstechnica.com·1d ago
Columbia researcher's own paper nearly published with AI-hallucinated reference, highlighting growing academic integrity concern
A Columbia University associate professor, Maxim Topaz, who researches AI applications in healthcare, nearly published a scientific paper co
fortune.com·4d ago