Towards Mechanistically Understanding Why Memorized Knowledge Fails to Generalize in Large Language Model Finetuning
arXiv:2607.08393v1 Announce Type: cross Abstract: Fine-tuning LLMs to inject new knowledge faces a critical challenge: LLMs can quickly memorize new facts, yet fail to use them for downstream…
Read the full articleYou might also wanna read
Study reveals why in-context learning fails on complex specification-heavy tasks and how fine-tuning can help
In-context learning (ICL) has become the default method for using large language models (LLMs), making the exploration of its limitations an
arxiv.org·19d ago
Localizing Factual Recall Circuits in Gemma Models via Activation Patching
Activation patching reveals how facts are stored, routed, and read out across transformer layers, and why the residual stream does most of t
towardsdatascience.com·11d ago
Comprehensive Survey of Reasoning Failures in Large Language Models
Large Language Models (LLMs) have exhibited remarkable reasoning capabilities, achieving impressive results across a wide range of tasks. De
arxiv.org·4mo ago
Study Reveals LLMs' Simulated Reasoning Abilities Are Fragile and Limited
Chain-of-thought AI “degrades significantly” when asked to generalize beyond training.
arstechnica.com·11mo ago
Ouro: Looped Language Models That Build Reasoning into Pre-Training Through Latent Space Iteration
Modern LLMs are trained to "think" primarily via explicit text generation, such as chain-of-thought (CoT), which defers reasoning to post-tr
arxiv.org·6mo ago
Latent learning: How episodic memory could improve machine learning generalization
When do machine learning systems fail to generalize, and what mechanisms could improve their generalization? Here, we draw inspiration from
openreview.net·27d ago
Comments
Sign in to join the conversation.
No comments yet. Be the first.