Ever heard of the lottery ticket hypothesis?
Our new paper shows that lottery tickets are not a useful metaphor to explain the success of overparameterized neural networks - and suggests an alternative metaphor: escape dimensions
This is cool and makes a lot of sense. Reminds me of the theory of neutral networks in evolutionary theory, where networks of neutral genotype changes enable populations to traverse the fitness landscape without getting stuck in local minima
GerstnerLab
Dr Nick
NEW PAPER. Why do larger networks train better?
"Because they contain more candidate *sub*networks that can learn the task" → lottery tickets
This popular explanation uses an appealing but misleading metaphor🧵
We propose an intuitive alternative grounded in theory: escape dimensions
Flavio Martinelli
NEW PAPER. Why do larger networks train better?
"Because they contain more candidate *sub*networks that can learn the task" → lottery tickets
This popular explanation uses an appealing but misleading metaphor🧵
We propose an intuitive alternative grounded in theory: escape dimensions
