Research map / Optimization / Optimizers
Aggregated Momentum: Stability Through Passive Damping
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use aggregated momentum instead of tuned classical or Nesterov momentum | MNIST autoencoder · MNIST autoencoder | validation loss: 3.20 -> 3.05 | Helped |
| Use AggMo with default settings instead of classical momentum at β=0.9 | MNIST autoencoder · MNIST autoencoder, default settings | validation loss: 4.95 -> 3.14 | Helped |
| Use AggMo instead of the strongest listed baseline | CNN-5 · CIFAR-10 | test accuracy: 64.32 -> 65.09 | Helped |
| Use AggMo instead of classical momentum | ResNet-32 · CIFAR-10 | validation accuracy: 94.20 -> 93.87 | Hurt |
| Use AggMo instead of Nesterov momentum | ResNet-32 · CIFAR-10 | test accuracy: 93.18 -> 93.16 | No clear effect |
| Use AggMo instead of classical momentum | ResNet-32 · CIFAR-100 | test accuracy: 70.21 -> 70.11 | Hurt |
| Use AggMo instead of Adam | Penn Treebank LSTM · Penn Treebank | test perplexity: 57.83 -> 57.79 | No clear effect |
| Use several averaged velocities rather than one high-damping velocity | AggMo · MNIST autoencoder optimization | optimization stability | Helped |
| Increase AggMo's maximum damping coefficient to 0.999 | AggMo · MNIST autoencoder optimization | convergence speed | Helped |
| Remove batch normalization and train with AggMo (ablation) | ResNet-32 · CIFAR-100 | test error: 67.26% -> 69.32% | Hurt |
| Use AggMo rather than CM for quadratic objectives in the under-damped regime | Momentum optimization · Quadratics with varying condition number | convergence rate: approximately three-times faster | Helped |
Closest papers by meaning
- Analytical Study of Momentum-Based Acceleration Methods in Paradigmatic High-Dimensional Non-Convex Problems (NeurIPS 2021 · similarity 0.86)
- On the Performance Analysis of Momentum Method: A Frequency Domain Perspective (ICLR 2025 · similarity 0.86)
- The Marginal Value of Momentum for Small Learning Rate SGD (ICLR 2024 · similarity 0.85)
- A geometric framework for momentum-based optimizers for low-rank training (NeurIPS 2025 · similarity 0.85)
- Gradient correlation is a key ingredient to accelerate SGD with momentum (ICLR 2025 · similarity 0.85)
- Perfect Parallelization in Mini-Batch SGD with Classical Momentum Acceleration (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.