AI Paper Atlas

Research map / Optimization / Optimizers

PASO: Step Parallel Stochastic Optimization

Jianrong Lu, Zhuoya Gu, Haobo Li, Zhiyu Zhu, Yechao Zhang, Jianhai Chen, Minghui Yang, Junwei Liu, Jian Wang, Qinming He, Hui LIU, Junhui Hou

ICML 2026 · Optimization › Optimizers · approach: Adam convergence and adaptive learning rates

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add PASO step-parallel solving to Adam/AdamWGPT-2 on WikiText-2 · WikiText-2 (pre-trained)iterations: 1000 -> 48 (20.8×)Improved
add PASO to AdamGPT-2 on WikiText-2 · WikiText-2 (pre-trained)test perplexity: 20.4 -> 20.7No clear effect
add PASO to AdamResNet50 on CIFAR-10 · CIFAR-10 (from scratch)top-1 accuracy: 90.1 -> 89.5Hurt
add PASO to AdamWViT on CIFAR-10 · CIFAR-10 (from scratch)top-1 accuracy: 78.0 -> 78.4No clear effect
add PASO to Adam for diffusion trainingUNet on LSUN Church · LSUN Church 128x128FID: 9.08 (600k iters) -> 9.55 (136k iters)Mixed
increase window size p from 7 to 21 (ablation)AdamW+PASO on GPT-2 · WikiText-2iterations: 143 -> 48Improved
increase window size p from 7 to 21 (ablation)AdamW+PASO on GPT-2 · WikiText-2wall-clock speedup: 4.0× -> 4.4×No clear effect
increase batch size from 52 to 112 (ablation)AdamW+PASO on GPT-2 · WikiText-2wall-clock speedup: 3.2× -> 4.4×Improved
vary tolerance delta in [1e-7, 0.1] and EMA decay in [0.9, 0.9999] (ablation)PASO with Adam on CIFAR-10 · CIFAR-10iterations and qualityNo clear effect
vary learning rate from 4e-4 to 1e-2 (ablation)PASO on CNN CIFAR-10 · CIFAR-10accuracyNo clear effect
use PASO step parallelism instead of unoptimized data parallelismGPT-2 Adam · WikiText-2wall-clock time: 381s -> 139sImproved
use PASO prototype instead of optimized DDP/DeepSpeedGPT-2 Adam · WikiText-2wall-clock time: 139s vs 140s (DDP), 138s (DeepSpeed)No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy