AI Paper Atlas

Research map / Optimization / Optimizers

Delving into Muon and Beyond: Deep Analysis and Extensions

Xianbiao Qi, Marco Chen, Jiaquan Ye, Yelin He, Rong Xiao

ICML 2026 · Optimization › Optimizers · approach: Adam convergence and adaptive learning rates · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
apply Muon orthogonalization (p=0) to momentum instead of raw momentum SGDmSGD · OpenWebText val lossvalidation loss: 3.092 -> 2.887Improved
use moderate spectral compression p=1/4 instead of full orthogonalization on momentum (ablation)mSGDZ (Muon) · OpenWebText val lossvalidation loss: 2.887 -> 2.876Improved
use p=1/2 instead of p=1 on momentum (ablation)mSGD · OpenWebText val lossvalidation loss: 3.092 -> 2.895Improved
add RMS normalization before the update instead of momentum input (ablation)Muon (p=0) · OpenWebText val lossvalidation loss: 2.887 -> 2.879Improved
add full orthogonalization (p=0) to RMS-normalized input (ablation)Adam · OpenWebText val lossvalidation loss: 2.871 -> 2.879Hurt
add mild compression p=1/2 to RMS-normalized input (ablation)Adam · OpenWebText val lossvalidation loss: 2.871 -> 2.870No clear effect
add p=1/4 compression to RMS-normalized input (ablation)Adam · OpenWebText val lossvalidation loss: 2.871 -> 2.872No clear effect
raise learning rate from 1e-1 to 10.0 beyond the optimum (ablation)mSGD · OpenWebText val lossvalidation loss: 3.649 -> 3.164Mixed
use Muon orthogonalization instead of mSGD for lr robustness (ablation)momentum optimizers · OpenWebText val lossvalidation loss: 3.649 -> 2.891Improved
add spectral compression p=1/2 to RMS-normalized input at small lr (ablation)Adam · OpenWebText val lossvalidation loss: 2.884 -> 2.918Mixed

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy