AI Paper Atlas

Research map / Optimization / Optimizers

An Exploration of Non-Euclidean Gradient Descent: Muon and its Many Variants

Michael Crawshaw, Chirag Modi, Mingrui Liu, Robert M. Gower

ICML 2026 · Optimization › Optimizers · approach: Adam convergence and adaptive learning rates

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use stale nuclear norm approximation (ablation)MuonMax · FineWeb 1B tokensfinal validation loss: 3.791 -> 3.768Improved
use stale nuclear norm approximation (ablation)MuonAdam-Momo · FineWeb 1B tokensfinal validation loss: 3.551 -> 3.554No clear effect
use stale nuclear norm approximation (ablation)Scion-Momo · FineWeb 1B tokensfinal validation loss: 3.592 -> 3.590No clear effect
use stale nuclear norm approximation (ablation)MuonMax-Momo · FineWeb 1B tokensfinal validation loss: 3.576 -> 3.580No clear effect
use stale nuclear norm approximation (ablation)MuonMax-Momo · FineWeb 1B tokensextra wall-clock time vs MuonAdam: 11% -> 5%Improved
add Momo model-based step sizeMuonAdam · FineWeb 1B tokensfinal validation loss: 3.604 -> 3.551Improved
add Momo model-based step sizeMuonMax · FineWeb 1B tokensfinal validation loss: 3.791 -> 3.576Improved
add Momo model-based step sizeMuonAdam · FineWeb 1B tokens, LR sweepproportion of LRs with loss < 3.65 (%): 25% -> 62.5%Improved
use MuonMax-Momo instead of Muon-Adam or ScionMuonMax-Momo · SlimPajama 1B tokens (GPT2-Large)LR robustness: 3.13 to 3.24Improved
use MuonMax-Momo instead of MuonAdamMuonMax-Momo · SlimPajama 6B tokens (GPT2-Large)final validation lossImproved
lower the Momo loss lower bound F* from 3.2 to 0.0 (ablation)MuonMax-Momo · FineWeb 1B tokensfinal validation loss: 3.58 -> 3.61Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy