Research map / Reinforcement learning / RL from feedback
In-The-Flow Agentic System Optimization for Effective Planning and Tool Use
Zhuofeng Li, Haoxiang Zhang, Seungju Han, Sheng Liu, Jianwen Xie, Yu Zhang, Yejin Choi, James Zou, Pan Lu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train the planner on-policy with Flow-GRPO (vs frozen planner) (ablation) | AgentFlow (Qwen2.5-7B) · 6-task average (Bamboogle, 2Wiki, GAIA, AIME24, AMC23, GameOf24) | accuracy: 38.5 -> 55.7 | Improved |
| replace frozen Qwen2.5-7B planner with frozen GPT-4o (ablation) | AgentFlow · 6-task average | accuracy: 38.5 -> 44.3 | Improved |
| offline SFT on GPT-4o trajectories for the planner (ablation) | AgentFlow · 6-task average | accuracy: 38.5 -> 19.5 | Hurt |
| add Flow-GRPO to AgentFlow with 7B backbone | AgentFlow · 2Wiki | accuracy: 60.0 -> 77.2 | Improved |
| add Flow-GRPO to AgentFlow | AgentFlow · GameOf24 | accuracy: 31.0 -> 53.0 | Improved |
| add Flow-GRPO to AgentFlow | AgentFlow · AMC23 | accuracy: 47.4 -> 61.5 | Improved |
| add Flow-GRPO to AgentFlow | AgentFlow · GPQA / MedQA | accuracy: GPQA 37.0 -> 47.0; MedQA 76.0 -> 80.0 | Improved |
| apply Flow-GRPO fine-tuning to both 3B and 7B backbones (ablation) | AgentFlow · multiple tasks | accuracy | Improved |
| increase maximum turns at inference from 3 to 10 (ablation) | AgentFlow (7B) · 2Wiki, GameOf24, AIME24, GAIA | accuracy: avg turns on GAIA 2.43 -> 5.42 | Improved |
| use Flow-GRPO-trained planner instead of untrained one (ablation) | AgentFlow · MedQA tool use | Google Search call ratio: 66.2% -> 10.9% Google Search; Wikipedia Search 0 -> 59.8% | Improved |
| compare Flow-GRPO-trained AgentFlow with monolithic tool RL (ToRL) | AgentFlow vs ToRL · AIME24 validation during training | validation accuracy | Improved |
Closest papers by meaning
- AstraFlow: Dataflow-Oriented Reinforcement Learning for Agentic LLMs (NeurIPS 2026 · similarity 0.91)
- Agentic Reinforced Policy Optimization (ICLR 2026 · similarity 0.91)
- Learning Agentic Policy from Action Guidance (NeurIPS 2026 · similarity 0.90)
- Tools as Continuous Flow for Evolving Agentic Reasoning (NeurIPS 2026 · similarity 0.89)
- Group-Graph Policy Optimization for Long-Horizon Agentic Reinforcement Learning (NeurIPS 2026 · similarity 0.89)
- Generalizable End-to-End Tool-Use RL with Synthetic CodeGym (ICLR 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
