Research map / Optimization / Optimizers
Shampoo: Preconditioned Stochastic Tensor Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Shampoo's tensor-structured preconditioning instead of commonly used optimizers | Deep-learning models · CIFAR-10, CIFAR-100, and LM1B | convergence speed | Helped |
| Recompute preconditioner roots every 20--100 steps rather than at every step (ablation) | Shampoo · CIFAR-10, CIFAR-100, and LM1B | amortized runtime per step | Helped |
| Recompute preconditioner roots every 20--100 steps rather than at every step (ablation) | Shampoo · CIFAR-10, CIFAR-100, and LM1B | accuracy | No clear effect |
| Add momentum with α=0.9 (ablation) | Shampoo · CIFAR-10, CIFAR-100, and LM1B | convergence | Helped |
| Use diagonal rather than full preconditioners where a dimension is too large (ablation) | Shampoo · Deep-learning experiments | performance | No clear effect |
Closest papers by meaning
- Purifying Shampoo: Investigating Shampoo's Heuristics by Decomposing its Preconditioner (NeurIPS 2025 · similarity 0.91)
- A New Perspective on Shampoo's Preconditioner (ICLR 2025 · similarity 0.91)
- Structured Preconditioners in Adaptive Optimization: A Unified Analysis (ICML 2025 · similarity 0.89)
- On The Concurrence of Layer-wise Preconditioning Methods and Provable Feature Learning (ICML 2025 · similarity 0.87)
- 4-bit Shampoo for Memory-Efficient Network Training (NeurIPS 2024 · similarity 0.86)
- Nonlinearly Preconditioned Gradient Methods: Momentum and Stochastic Analysis (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.