The Bitter Lesson: Why Computation Beats Human Knowledge in AI Research
Reliable enough to start your morning with. Toast it again tomorrow.
Summary
Rich Sutton argues that the key lesson from 70 years of AI research is that general methods leveraging massive computation ultimately outperform approaches that rely on encoding human knowledge. He illustrates this with examples from computer chess, Go, and speech recognition, where brute-force search and scalable computation defeated human-expert-crafted systems. Sutton contends that researchers who try to build human-like understanding into AI are fighting a losing battle against the exponential growth of computing power.
Key quotes
· 4 pulledThe biggest lesson that can be read from 70 years of AI research is that general methods that leverage computation are ultimately the most effective, and by a large margin.
The ultimate reason for this is Moore's law, or rather its generalization of continued exponentially falling cost per unit of computation.
These researchers wanted methods based on human input to win and were disappointed when they did not.
We have to learn the bitter lesson that building in our knowledge is a net negative in the long run, and that we must instead build systems that can discover their own knowledge.
You might also wanna read
Exploring the Limits of Rich Sutton's 'Bitter Lesson' in AI Development
The article discusses 'the bitter lesson,' a concept introduced by Rich Sutton, which emphasizes that general computational methods are the
dbreunig.com·10mo ago
Reflections on AI's Bitter Lesson and the Decline of Decision Theory
The author reflects on their experience writing about decision theory being abandoned in mainstream AI, only to be told by Hacker News comme
gfrm.in·3mo ago
Balancing Domain Knowledge and General-Purpose Methods in AI Research
The article discusses the concept of 'The Bitter Lesson' in AI research, highlighting the balance between domain knowledge and general-purpo
assaf-pinhasi.medium.com·10mo ago
The Bitter Lesson of AI Product Development: Why General Methods Win
The article discusses the "Bitter Lesson" from AI product development - that general methods leveraging computation ultimately prove most ef
hex.tech·8mo ago
Examining the Limitations of Transformer Models and the Gap to Human-Level AI
The article presents a skeptical perspective on claims about imminent Artificial General Intelligence (AGI), arguing that current transforme
dlants.me·3mo ago
AI Systems Will Become Less Human-Like as They Scale, Despite Superficial Similarities
The article argues that while modern AI systems like LLMs appear remarkably human in their conversational abilities and imperfections, this
research.roundtable.ai·5mo ago