Research map / Optimization / Implicit bias and generalization
How Muon’s Spectral Design Benefits Generalization: A Study on Imbalanced Data
Bhavya Vasudeva, Puneesh Deora, Yize Zhao, Vatsal Sharan, Christos Thrampoulidis
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use spectral norm update (Muon) instead of Frobenius (NMD) or max-norm (Signum) updates | MLP · Colored-MNIST (99% digit-colour correlation), minority groups | test accuracy | Improved |
| use Muon instead of NMD and Signum | ResNet-18/50 · CIFAR-10/100 STEP imbalance 20:1, minority classes | test accuracy | Improved |
| use SpecGD instead of GD (early stopped) | linear model, squared loss · Gaussian mixture, class imbalance | minority-class and balanced loss gap: minority gap >= mu t/4, balanced gap >= mu t/2 | Improved |
| use SpecGD instead of normalized GD | linear model, squared loss · Gaussian mixture, class imbalance | minority-class and balanced loss gap: gaps >= mu t/2 | Improved |
| use SpecGD instead of NGD/SignGD (early stopped) | linear model, cross-entropy · heavy-tailed imbalance, 20 classes | class-balanced and worst-class accuracy | Improved |
| increase depth of SpecGD-trained linear model | SpecGD · deep linear model | gap between component saturation times | Improved |
| use Muon instead of SGD | ResNet-34 · Dominoes MNIST-CIFAR, worst-group accuracy | worst-group and decoded worst-group accuracy | Improved |
| use Muon instead of Adam | ResNet-34 · Dominoes MNIST-CIFAR, worst-group accuracy | worst-group accuracy | No clear effect |
| use Muon/Shampoo instead of SGD | BERT-base · MultiNLI fine-tuning | worst-group accuracy | Improved |
| use Muon instead of Adam | BERT-base · MultiNLI fine-tuning | worst-group accuracy | Hurt |
| use Muon instead of Adam | ResNet-50 · CelebA fine-tuning | worst-group accuracy | No clear effect |
| use Muon instead of SGD and Adam | small Transformer · TinyStories rare and frequent tokens | top-1 and top-5 accuracy | Improved |
Closest papers by meaning
- Phases of Muon: When Muon Eclipses SignSGD (NeurIPS 2026 · similarity 0.89)
- Implicit Bias of Spectral Descent and Muon on Multiclass Separable Data (NeurIPS 2025 · similarity 0.88)
- Muon Dynamics as a Spectral Wasserstein Flow (NeurIPS 2026 · similarity 0.86)
- From SGD to Muon: Adaptive Optimization via Schatten-p Norms (NeurIPS 2026 · similarity 0.86)
- On the Convergence Analysis of Muon (NeurIPS 2026 · similarity 0.86)
- Tensorion: A Tensor-Aware Generalization of the Muon Optimizer (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
