Interested in backprop-free evolution suitable for modern billion parameter models at large population sizes? Super excited to share our latest work, and big congrats to my collaborators @bidiptas13.bsky.social, @juanduquevan.bsky.social and the rest of the @flair-ox.bsky.social team :)
Mattie Fellows
Introducing 🥚EGGROLL 🥚(Evolution Guided General Optimization via Low-rank Learning)! 🚀 Scaling backprop-free Evolution Strategies (ES) for billion-parameter models at large population sizes
âš¡100x Training Throughput
🎯Fast Convergence
🔢Pure Int8 Pretraining of RNN LLMs
