Unpacking the Grokking Transition: A Deep Dive into Compression Delays in Neural Networks
Exploring how modular arithmetic reveals key insights into neural network embeddings, with compression lagging behind generalization by extensive steps.
Read the full articleYou might also wanna read
Theoretical Analysis Reveals Why Linear RNNs Are More Parallelizable Than Nonlinear RNNs
The community is increasingly exploring linear RNNs (LRNNs) as language models, motivated by their expressive power and parallelizability. W
arxiv.org·1mo ago
Wider Neural Networks with Fewer Parameters Improve Performance by Reducing Feature Interference
This work demonstrates how increasing the number of neurons in a network without increasing its total number of non-zero parameters improves
arxiv.org·1mo agoEmergence of Diffusion Models from Associative Memory
Article URL: Comments URL: Points: 3 # Comments: 0
New preprint:
New preprint: Random network structure stabilizes neural manifolds We’re excited to share our new work on representational drift. doi.org/10
Theoretical Foundations of Deep Learning
Theoretical Foundations of Deep Learning
clcoding.com·24d ago
Emergent Hebbian Dynamics in Regularized Learning: A Theoretical Analysis
Hebbian and anti-Hebbian plasticity are widely observed in the brain and are classically modeled as mechanistic, local homosynaptic rules st
arxiv.org·1mo ago
Comments
Sign in to join the conversation.
No comments yet. Be the first.