AI Paper Atlas

Research map / Optimization / Implicit bias and generalization

Egalitarian Gradient Descent: A Simple Approach to Accelerated Grokking

Ali Saheb Pasand, Elvis Dohmatob

ICLR 2026 · Optimization › Implicit bias and generalization · approach: Edge-of-stability dynamics · 2 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
replace vanilla SGD with EGD (exact SVD)two-layer MLP · Modular addition (p=79)epochs to 95% test accuracy: 10099 -> 222Improved
replace vanilla SGD with EGD (exact SVD)two-layer MLP · Modular multiplication (p=97)epochs to 95% test accuracy: 5715 -> 107Improved
replace vanilla SGD with EGD (exact SVD)two-layer MLP · Sparse parity (k=2)epochs to 95% test accuracy: 7873 -> 109Improved
use randomized SVD with rank 128 instead of exact SVD (ablation)EGD · Modular addition (p=127)epochs to 95% test accuracy: 62 -> 76 epochs; 11.4 -> 10.1 sMixed
use randomized SVD rank 40 instead of exact SVD (ablation)EGD · Sparse parity (k=2)epochs to 95% test accuracy: 109 -> 42Improved
use randomized SVD rank 40 instead of exact SVD (ablation)EGD · Sparse parity (k=4)epochs to 95% test accuracy: 254 -> -Hurt
reduce RSVD rank from 40 to 10 (ablation)EGD · Sparse parity (k=2)epochs to 95% test accuracy: 42 -> 2586Hurt
use RSVD rank 30 instead of exact SVD (ablation)EGD · Sparse parity (k=3)epochs to 95% test accuracy: 115 -> 76Improved
use column normalization (SVD-free) instead of exact SVD (ablation)EGD · Modular addition (p=127)epochs to 95% test accuracy: 62 -> 133 epochs; 11.4 -> 5.9 sMixed
use column normalization instead of exact SVD (ablation)EGD · Sparse parity (k=4)epochs to 95% test accuracy: 254 -> -Hurt
use best-rank RSVD instead of exact SVD, average over tasks (ablation)EGD · Average over all tasksepochs to 95% test accuracyImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy