The Latency Problem with Natural Language Interfaces: Why LLMs Aren't Always the Right Choice
By
steveklabnik
4mo ago· 6 min readenInsight
100/100
Golden Brown
Bagelometer↗
Hand-rolled, kettle-boiled, baked to perfection. Worth every minute at the bakery.
Score100TypeanalysisSentimentneutral
Summary
The article argues against the indiscriminate use of natural language interfaces powered by LLMs, highlighting their significant latency and cost issues compared to traditional graphical user interfaces. It presents a technical critique of LLM inference speeds, noting that natural language interfaces have orders of magnitude more latency than standard GUIs. The author acknowledges that natural language is a wonderful interface but cautions that just because we can use LLMs doesn't mean we always should, advocating for smarter interface design that balances the benefits of natural language with practical performance considerations.
Key quotes
· 4 pulledNatural language is a wonderful interface, but just because we suddenly can doesn't mean we always should.
LLM inference is slow and expensive, often taking tens of seconds to complete.
Natural language interfaces have orders of magnitude more latency than normal graphic user interfaces.
This doesn't mean we shouldn't use LLMs, it just means we need to be smart about how we build interfaces around them.
Natural language is a wonderful interface, but just because we suddenly can doesn't mean we always should.
