Research map / Reinforcement learning / Visual and representation RL
RLlib Flow: Distributed Reinforcement Learning is a Dataflow Problem
Eric Liang, Zhanghao Wu, Michael Luo, Sven Mika, Joseph E. Gonzalez, Ion Stoica
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| port to RLlib Flow dataflow model (minimal count) | RLlib IMPALA · IMPALA | lines of code: 694 -> 89 | Improved |
| port to RLlib Flow (conservative count incl. shared operators) | RLlib IMPALA · IMPALA | lines of code: 694 -> 362 | Improved |
| port to RLlib Flow | RLlib A3C · A3C | lines of code: 87 -> 11 | Improved |
| port to RLlib Flow | RLlib A2C · A2C | lines of code: 154 -> 25 | Improved |
| port to RLlib Flow | RLlib DQN · DQN | lines of code: 239 -> 87 | Improved |
| port to RLlib Flow | RLlib PPO · PPO | lines of code: 386 -> 79 | Improved |
| port to RLlib Flow | RLlib Ape-X · Ape-X | lines of code: 250 -> 126 | Improved |
| port to RLlib Flow | original MAML · MAML | lines of code: 370 -> 139 | Improved |
| count shared operators in RLlib Flow lines of code | RLlib Flow · Ape-X | code reduction ratio: 1.9 -> 1.1 | Hurt |
| implement PPO in RLlib Flow instead of Spark Streaming | PPO · CartPole-v0 distributed | training time: up to 2.9x | Improved |
| use RLlib Flow instead of original RLlib implementation | RLlib · Atari throughput, IMPALA, CartPole | throughput / sample efficiency | No clear effect |
Closest papers by meaning
- SRL: Scaling Distributed Reinforcement Learning to Over Ten Thousand Cores (ICLR 2024 · similarity 0.91)
- AstraFlow: Dataflow-Oriented Reinforcement Learning for Agentic LLMs (NeurIPS 2026 · similarity 0.91)
- Cleanba: A Reproducible and Efficient Distributed Reinforcement Learning Platform (ICLR 2024 · similarity 0.88)
- HyperMARL: Adaptive Hypernetworks for Multi-Agent RL (NeurIPS 2025 · similarity 0.86)
- RL-GPT: Integrating Reinforcement Learning and Code-as-policy (NeurIPS 2024 · similarity 0.86)
- Random Policy Evaluation Uncovers Policies of Generative Flow Networks (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
