Why LLM Evaluation Methods Fail When Models Enter New Capability Regimes

Summary

The article argues that current evaluation methods for LLMs are fundamentally flawed because they assume future models will be incremental improvements on current ones. When models cross into new capability regimes (becoming "different kinds of things"), existing benchmarks, safety evals, and red-teaming protocols break silently without detection. The author identifies this as the most important unsolved problem in understanding LLMs and suggests that the solution lies in evaluation methodology itself, not in training approaches.