Impact of Increasing Input Tokens on LLM Performance
Recent developments in large language models (LLMs) are focusing on longer context windows with millions of input tokens. The assumption that these models perform uniformly well across long-context tasks, based on benchmarks like Needle in a Haystack (NIAH), may not hold true. NIAH primarily evaluates simple retrieval tasks within extensive text documents.
research.trychroma.com10mo ago
