Research map / Optimization / Optimizers
MuonBP: Faster Muon via Block-Periodic Orthogonalization
Ahmed Khaled, Kaan Ozkara, Tao Yu, Mingyi Hong, Youngsuk Park
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add periodic full orthogonalization to BlockMuon (MuonBP) (ablation) | BlockMuon · 960M validation perplexity | validation perplexity: 20.29 -> 15.12 | Improved |
| add periodic full orthogonalization to BlockMuon (MuonBP) (ablation) | BlockMuon · 8B large LR | validation perplexity: 24.68 -> 12.97 | Improved |
| block-periodic orthogonalization instead of full each step | Muon · 1.2B | validation perplexity: 14.13 -> 13.78 | Improved |
| block-periodic orthogonalization instead of full each step | Muon · 8B | validation perplexity: 12.90 -> 12.77 | Improved |
| block-periodic orthogonalization instead of full each step | Muon · 8B throughput | throughput (TFLOP/s/GPU): 105.09 -> 113.37 | Improved |
| block-periodic orthogonalization instead of full each step | Muon · 960M throughput | throughput (TFLOP/s/GPU): 112.97 -> 113.54 | No clear effect |
| use Muon instead of AdamW | AdamW · 960M | validation perplexity: 22.51 -> 15.33 | Improved |
| use Muon instead of AdamW | AdamW · 8B | validation perplexity: 14.47 -> 12.90 | Improved |
| decrease the block period (ablation) | MuonBP · 280M FineWeb across TP degrees | validation loss | Improved |
| use MuonBP instead of Dion | Dion · 160M FineWeb | min validation loss: 3.37 -> 3.34 | Improved |
| use MuonBP instead of Dion | Dion · 160M throughput | throughput (TFLOP/s/GPU): 45.64 -> 51.40 | Improved |
| use MuonBP instead of Muon | Muon · 160M FineWeb | min validation loss: 3.36 -> 3.34 | Improved |
Closest papers by meaning
- Towards Understanding the Power and Limits of the Muon Optimizer: A River-Valley Perspective (NeurIPS 2026 · similarity 0.90)
- AMUSE: Anytime Muon with Stable Gradient Evaluation (NeurIPS 2026 · similarity 0.89)
- Powering Up Zeroth-Order Training via Subspace Gradient Orthogonalization (NeurIPS 2026 · similarity 0.88)
- Sharp Capacity Scaling of Spectral Optimizers in Learning Associative Memory (NeurIPS 2026 · similarity 0.87)
- Tensorion: A Tensor-Aware Generalization of the Muon Optimizer (NeurIPS 2026 · similarity 0.87)
- MuLoCo: Muon is a Practical Inner Optimizer for DiLoCo (ICML 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
