How Typing Habits Like Typos and Filler Words Inflate AI Token Counts

Summary

This article explores how human typing habits—such as typos, shorthand, filler words, pasted IDs, and stray whitespace—can significantly impact token counts when using AI language model tokenizers. The author demonstrates this with a simple example: a 5-word prompt with 2 spelling mistakes used 13 tokens, while the corrected version used only 6 tokens. The article compares tokenization behavior between OpenAI and Claude's tokenizers, noting that Claude tends to produce more tokens for the same text. The core insight is that ordinary typing habits can change token counts without substantially changing intent, which has cost implications since providers bill per token.