Research map / Reinforcement learning / RL from feedback
DistFlow: A Fully Distributed RL Framework for Scalable and Efficient LLM Post-Training
Zhixin Wang, Jiaming Xu, Tianyi Zhou, Mingjun Zhang, Liming Liu, Jiarui Hu, Dian Yang, TongYu Wang, Ping Zhang, Jinlong Hou, Siyuan Feng, Yuan Qi and 1 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Local Cache to the baseline data pipeline (ablation) | DistFlow · Ablation, batch size 2048 | speedup: 1.38x speedup at Batch Size=2048 | Improved |
| add Double Buffering with asynchronous reset (ablation) | DistFlow · Ablation, batch size 2048 | speedup: 1.38x to 1.45x | Improved |
| add Load Balancing (Constrained LPT) (ablation) | DistFlow · Ablation, batch size 4096 | speedup: 1.38x to 1.58x | Improved |
| add Load Balancing (Constrained LPT) (ablation) | DistFlow · Ablation, batch size 1024 | speedup | No clear effect |
| add Distributed Data Buffer (ablation) | DistFlow · Ablation, batch sizes 2048 and 4096 | speedup: peak speedup of 1.70x (Batch Size=2048) and 1.69x (Batch Size=4096) | Improved |
| increase context length from 8k to 64k | DistFlow vs verl, 7B · Long-context GRPO | speedup over verl: 1.48x at an 8k context length to 2.03x at 64k | Improved |
| use fully distributed DistFlow instead of centralized verl, GRPO instead of PPO | DistFlow · End-to-end throughput | speedup over verl: 1.09x - 1.64x for PPO; up to 2.62x for GRPO | Improved |
| replace verl with DistFlow for 32B GRPO training | DistFlow · Convergence, DeepScaleR, 32 GPUs | total execution time: reducing total execution time by 21% | Improved |
Closest papers by meaning
- SRL: Scaling Distributed Reinforcement Learning to Over Ten Thousand Cores (ICLR 2024 · similarity 0.90)
- AstraFlow: Dataflow-Oriented Reinforcement Learning for Agentic LLMs (NeurIPS 2026 · similarity 0.90)
- AREAL: A Large-Scale Asynchronous Reinforcement Learning System for Language Reasoning (NeurIPS 2025 · similarity 0.89)
- DORA: A Scalable Asynchronous Reinforcement Learning System for Language Model Training (NeurIPS 2026 · similarity 0.88)
- Highly Parallelized Reinforcement Learning Training with Relaxed Assignment Dependencies (AAAI 2025 · similarity 0.87)
- Trajectory Balance with Asynchrony: Decoupling Exploration and Learning for Fast, Scalable LLM Post-Training (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
