Decoding Infinite-Width Neural Networks: Breaking Down the Myths

Infinite-width neural networks offer insights but don't always match finite networks' biases. A new theorem challenges our assumptions.

You might also wanna read

This work demonstrates how increasing the number of neurons in a network without increasing its total number of non-zero parameters improves

The community is increasingly exploring linear RNNs (LRNNs) as language models, motivated by their expressive power and parallelizability. W

Lotteries and tickets are often used as a didactical analogy to explain the success of overparameterized neural networks: “larger networks s

Hebbian and anti-Hebbian plasticity are widely observed in the brain and are classically modeled as mechanistic, local homosynaptic rules st

We show that deep neural networks trained across diverse tasks exhibit remarkably similar low-dimensional parametric subspaces. We provide t

No comments yet. Be the first.