Research map / Reinforcement learning / RL from feedback
Information Gain-based Policy Optimization: A Simple and Effective Approach for Multi-Turn LLM Agents
Guoqing Wang, Sunhao Dai, Guangze Ye, Zeyu Gan, Wei Yao, Yong Deng, Xiaofeng Wu, Zhenzhe Ying
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add turn-level information gain reward to outcome reward (ablation) | GRPO on Qwen2.5-7B-Instruct · 7 QA datasets avg | F1: 51.9 -> 60.2 | Improved |
| add turn-level information gain reward to outcome reward (ablation) | GRPO on Qwen2.5-3B-Instruct · 7 QA datasets avg | F1: 32.3 -> 48.9 | Improved |
| use information gain reward only, no outcome reward (ablation) | IGPO on Qwen2.5-7B-Instruct · 7 QA datasets avg | F1: 53.5 vs 51.9 (GRPO), 60.2 (full) | Mixed |
| use information gain only vs outcome F1 only (ablation) | IGPO on Qwen2.5-3B-Instruct · 7 QA datasets avg | F1: 32.3 -> 34.6 | Improved |
| separate normalization of IG and outcome rewards vs joint (ablation) | IGPO on Qwen2.5-7B-Instruct · 7 QA datasets avg | F1: 58.7 -> 59.4 | Improved |
| use log-probability instead of probability for information gain (ablation) | IGPO on Qwen2.5-7B-Instruct · 7 QA datasets avg | F1: 59.4 -> 60.2 | Improved |
| use log-probability instead of probability for information gain (ablation) | IGPO on Qwen2.5-3B-Instruct · 7 QA datasets avg | F1: 48.0 -> 48.9 | Improved |
| IGPO instead of GRPO | Qwen2.5-7B-Instruct · Musique | F1: 25.0 -> 32.7 | Improved |
| IGPO instead of GRPO/PPO/GSPO | Qwen2.5-7B-Instruct · NQ | F1: 40.3 -> 46.4 | Improved |
| IGPO instead of GiGPO step reward | Qwen2.5-7B-Instruct · 7 QA datasets avg | F1: 47.2 -> 60.2 | Improved |
| IGPO instead of GRPO | Qwen2.5-7B-Instruct · NQ vs GRPO | F1 | No clear effect |
| information gain reward adds computation | IGPO vs GRPO · training step latency | overhead: <0.02% end-to-end | No clear effect |
Closest papers by meaning
- Agentic Reinforced Policy Optimization (ICLR 2026 · similarity 0.92)
- Skill-SD: Skill-Conditioned Self-Distillation for Multi-turn LLM Agents (NeurIPS 2026 · similarity 0.91)
- Group-in-Group Policy Optimization for LLM Agent Training (NeurIPS 2025 · similarity 0.91)
- ArCHer: Training Language Model Agents via Hierarchical Multi-Turn RL (ICML 2024 · similarity 0.91)
- From Player to Master: Enhancing Test-Time Learning of LLM Agents via Reinforcement Learning over Memory (ICML 2026 · similarity 0.90)
- LMRL Gym: Benchmarks for Multi-Turn Reinforcement Learning with Language Models (ICML 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
