Research map / Reinforcement learning / RL from feedback
FlowRL: Matching Reward Distributions for LLM Reasoning
Xuekai Zhu, Daixuan Cheng, Dinghuai Zhang, Hengli Li, Kaiyan Zhang, Che Jiang, Youbang Sun, Ermo Hua, Yuxin Zuo, Xingtai Lv, Qizheng Zhang, Lin Chen and 11 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add importance sampling to FlowRL (ablation) | FlowRL · Math average (Avg@16), Qwen2.5-7B-Base | Avg@16 accuracy: 26.71 -> 35.63 | Improved |
| use trajectory-level IS instead of combined GFlowNets+PPO loss (ablation) | FlowRL · Math average (Avg@16), Qwen2.5-7B-Base | Avg@16 accuracy: 33.67 -> 35.63 | Improved |
| raise beta from 5 to 15 (ablation) | FlowRL · Math average (Avg@16), Qwen2.5-7B-Base | Avg@16 accuracy: 31.34 -> 35.63 | Improved |
| raise beta from 15 to 30 (ablation) | FlowRL · Math average (Avg@16), Qwen2.5-7B-Base | Avg@16 accuracy: 35.63 -> 35.09 | Hurt |
| raise beta from 10 to 15 (ablation) | FlowRL · Math average (Avg@16), Qwen2.5-7B-Base | Avg@16 accuracy: 34.41 -> 35.63 | Improved |
| replace GRPO with FlowRL | GRPO · Math average, Qwen2.5-32B-Base | Avg@16 accuracy: 38.34 -> 48.39 | Improved |
| replace GRPO with FlowRL | GRPO · Math average (Avg@16), Qwen2.5-7B-Base | Avg@16 accuracy: 32.48 -> 35.63 | Improved |
| replace PPO with FlowRL | PPO · Math average, Qwen2.5-32B-Base | Avg@16 accuracy: 43.25 -> 48.39 | Improved |
| replace GRPO with FlowRL | GRPO · LiveCodeBench | Avg@16: 32.75 -> 37.43 | Improved |
| replace GRPO with FlowRL | GRPO · CodeForces | rating: 1313.82 -> 1549.47 | Improved |
| replace GRPO with FlowRL | GRPO · HumanEval+ | Avg@16: 80.13 -> 83.28 | Improved |
| replace GRPO with FlowRL, temperature 1.0 | GRPO · Math average, Qwen2.5-7B (Avg@64, T=1.0) | Avg@64 accuracy: 32.44 -> 34.62 | Improved |
Later papers that compared with this method
6 later accepted papers report FlowRL as a baseline in their results tables; 5 of them report a better number on most of the results they share with it.
- FLAG: Flow Policy MaxEnt-RL by Latent Augmented Guidance (NeurIPS 2026 · better on 6 of 6 shared results)
- Beyond Normalization: Rethinking the Partition Function as a Difficulty Scheduler for RLVR (ICML 2026 · better on 5 of 5 shared results)
- Expected Return Causes Outcome-Level Mode Collapse in Reinforcement Learning and How to Fix It with Inverse Probability Scaling (ICML 2026 · better on 4 of 4 shared results)
- Mean Flow Policy Optimization (ICML 2026 · better on 1 of 2 shared results)
- Reparameterization Flow Policy Optimization (ICML 2026 · better on 2 of 2 shared results)
- Beyond Mode Collapse: Distribution Matching for Diverse Reasoning (ICML 2026 · better on 1 of 1 shared result)
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Flow of Reasoning: Training LLMs for Divergent Reasoning with Minimal Examples (ICML 2025 · similarity 0.91)
- Rectifying LLM Thought from Lens of Optimization (ICLR 2026 · similarity 0.91)
- SetPO: Set-Level Policy Optimization for Diversity-Preserving LLM Reasoning (ICML 2026 · similarity 0.90)
- Count Counts: Motivating Exploration in LLM Reasoning with Count-based Intrinsic Rewards (ICLR 2026 · similarity 0.90)
- Reinforcement Learning with Verifiable Rewards Implicitly Incentivizes Correct Reasoning in Base LLMs (ICLR 2026 · similarity 0.89)
- Consistent Paths Lead to Truth: Self-Rewarding Reinforcement Learning for LLM Reasoning (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
