AI Paper Atlas

Research map / Architectures and efficiency / Quantization

M+Adam: Low-Precision Training via Additive–Multiplicative Optimization

Xiaoyuan Liang, Sebastian Loeschcke, Mads Toftrup, Anima Anandkumar

ICML 2026 · Architectures and efficiency › Quantization · approach: Low-precision training formats · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add multiplicative update branch (M+Adam instead of AdamW)AdamW · C4/EN 1x Chinchilla BF16/BF16, 350Mvalidation perplexity: 19.156 -> 16.607Improved
add multiplicative update branch (M+Adam instead of AdamW)AdamW · C4/EN 1x Chinchilla BF16/BF16, 60Mvalidation perplexity: 29.865 -> 29.035Improved
add multiplicative branch instead of AdamWAdamW · C4/EN 1x Chinchilla NVFP4/FP8, 350Mvalidation perplexity: 24.941 -> 20.796Improved
add multiplicative branch instead of AdamWAdamW · C4/EN 1x Chinchilla NVFP4/FP8, 60Mvalidation perplexity: 33.884 -> 30.948Improved
add multiplicative branch instead of AdamWAdamW · C4/EN 1x Chinchilla FP8/FP8, 350Mvalidation perplexity: 21.674 -> 19.395Improved
add multiplicative branch instead of AdamWAdamW · C4/EN 1x Chinchilla BF16/FP8, 1Bvalidation perplexity: 16.346 -> 15.690Improved
use M+Adam instead of AdamW with stochastic roundingAdamW + SR · C4/EN 1x Chinchilla BF16/BF16, 350Mvalidation perplexity: 18.720 -> 16.607Improved
use M+Adam instead of AdamW with Kahan compensationAdamW + Kahan · C4/EN 1x Chinchilla BF16/BF16, 130Mvalidation perplexity: 22.859 -> 22.032Improved
use stochastic rounding with AdamWAdamW · C4/EN 1x Chinchilla BF16/BF16, 1Bvalidation perplexity: 15.642 -> 14.346Improved
compress multiplicative second-moment state with Apollo-E rank 4 (ablation)M+Adam · C4/EN 1x Chinchilla BF16/BF16, 350Mvalidation perplexity: 16.607 -> 16.550No clear effect
increase Apollo-E rank from 4 to 16 (ablation)M+Adam with Apollo-E · C4/EN 1x Chinchilla BF16/BF16, 130Mvalidation perplexity: 21.979 -> 21.995No clear effect
use mini tensor-wise Apollo-E instead of channel-wise rank 1 (ablation)M+Adam with Apollo-E · C4/EN 1x Chinchilla BF16/BF16, 350Mvalidation perplexity: 16.683 -> 16.752Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy