Investigating Why LLMs Falsely Claim a Seahorse Emoji Exists

Summary

This article investigates a peculiar phenomenon where multiple large language models (LLMs) including GPT-4, Claude Sonnet, and Gemini incorrectly claim that a seahorse emoji exists, despite there being no official seahorse emoji in the Unicode standard. The author conducts systematic testing across different models, revealing consistent false positives and explores the underlying reasons using tools like logitlens to understand how LLMs generate these confident but incorrect responses about emoji existence.