Research map / Learning theory and algorithms / Matrix and tensor methods

Intrinsic Muon: Spectral Optimization on Riemannian Matrix Manifolds

NeurIPS 2026 · Learning theory and algorithms › Matrix and tensor methods · approach: Low-rank approximation theory · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes intrinsic Muon (iMuon), an LMO-based optimizer using a metric-induced intrinsic unitarily invariant norm on tangent spaces, giving closed-form updates on fixed-rank, SPD, Stiefel and Grassmann manifolds with dimension-dependent convergence rates.

Problem

Extending norm-constrained Muon-style LMO optimizers to manifold-valued parameters (low-rank factorizations, Stiefel, SPD, Grassmann) fails due to quotient symmetry breaking and the coupling of tangent-space and ambient norm constraints, which obstructs closed-form solutions.

Approach

Lift any unitarily invariant Euclidean norm to an intrinsic tangent-space norm via the Riemannian metric (phi(G_x^{1/2} xi)); under a left-right metric form and singular-value invariance of the scaled tangent space, the intrinsic LMO reduces to a Euclidean SVD-based problem with closed-form solutions and symmetry preservation.

Key results

• On E2E NLG with GPT-2 Medium (LoRA r=4, no momentum), iMuon reaches 70.74 BLEU vs 70.02 for factor-wise Muon and Riemannion.
• On GLUE with Mistral-7B (LoRA r=16, 4-bit), iMuon averages 86.25 vs 81.68 for factor-wise Muon.
• On SPD classification on CIFAR-100 (S^32, 3 seeds), iMuon reaches 0.475 vs Muon 0.407 test accuracy; iMuon-Nu 0.546 vs NuMuon 0.422.

Limitations

• Requires left-right metric form and singular-value invariance of scaled tangent space, which hold only on the four treated manifolds.
• Uses an l_infinity product-norm relaxation of the joint norm on product tangent spaces such as fixed-rank and Stiefel.
• Stochastic convergence has an additive 2*sigma floor without variance reduction or momentum.

Experimental setup

Datasets: E2E NLG GLUE CIFAR-100

Benchmarks: E2E NLG GLUE CIFAR-100

Metrics: BLEU NIST METEOR ROUGE-L CIDEr accuracy

Baselines: SGD Scaled GD Factor-wise Muon Riemannion EGD Muon NuMuon

Builds on: Muon Riemannian preconditioned gradient descent Scaled GD NuMuon Riemannion

Closest papers by meaning

• Tensorion: A Tensor-Aware Generalization of the Muon Optimizer (NeurIPS 2026 · similarity 0.89)
• Phases of Muon: When Muon Eclipses SignSGD (NeurIPS 2026 · similarity 0.87)
• Muon Dynamics as a Spectral Wasserstein Flow (NeurIPS 2026 · similarity 0.86)
• Constrained Stochastic Spectral Preconditioning Converges for Nonconvex Objectives (NeurIPS 2026 · similarity 0.86)
• On the Convergence Analysis of Muon (NeurIPS 2026 · similarity 0.86)
• From SGD to Muon: Adaptive Optimization via Schatten-p Norms (NeurIPS 2026 · similarity 0.86)