The EGGROLL Recipe
🧠🛠️ We replace full-rank perturbations with low-rank ones. Each update is still high rank, maintaining expressivity with faster training
🥚 EGGROLL converges to the full-rank update at a fast rate of 1/rank. The method is effective even with a rank of 1
