Addressing the Limitations of Large Language Models with a Day-Dreaming Loop
By
nanfinitum
10mo ago· 14 min readenInsight
100/100
Golden Brown
Bagelometer↗
Pulled from the oven just right. Trustworthy, fact-dense, deeply satisfying.
Score100TypeanalysisSentimentneutral
Summary
The article discusses the limitations of large language models (LLMs) in producing genuine breakthroughs and proposes a day-dreaming loop (DDL) to address these limitations by introducing a background process for exploring non-obvious links between concepts.
Key quotes
· 3 pulledLarge language models lack some fundamental aspects of human thought: they are frozen, unable to learn from experience, and they have no 'default mode' for background processing.
I propose a day-dreaming loop (DDL): a background process that continuously samples pairs of concepts from memory.
A generator model explores non-obvious links between concepts, and a critic model filters the results for genuinely valuable ideas.
Proposal & discussion of how default mode networks for LLMs are an example of missing capabilities for search and novelty in contemporary AI systems.
