Research map / Reinforcement learning / RL from feedback
Long Live The Balance: Information Bottleneck Driven Tree-based Policy Optimization
Hao Jiang, Shurui Li, Tianpeng Bu, Bowen Xu, Xin Liu, Qihua Chen, hongtao duan, lulu hu, Bin Yang, Minying Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add IBTPO (IB-guided tree sampling and IB advantage) | GRPO on Qwen3-1.7B-Base · 7 benchmarks overall | avg@32 accuracy: 26.3% -> 29.2% | Improved |
| add IBTPO | GRPO on Qwen3-8B-Base · 7 benchmarks overall | avg@32 accuracy: 40.7% -> 44.3% | Improved |
| add clip-higher | GRPO on Qwen3-1.7B-Base · 7 benchmarks overall | avg@32 accuracy: 26.3% -> 26.0% | No clear effect |
| add entropy regularization | GRPO on Qwen3-1.7B-Base · 7 benchmarks overall | avg@32 accuracy: 26.3% -> 24.9% | Hurt |
| add IBTree sampling (ablation) | GRPO on Qwen3-8B-Base · AMC 24 | avg@32 accuracy: 39.4% -> 43.8% | Improved |
| add IB advantage alone (ablation) | GRPO on Qwen3-8B-Base · AIME 25 | avg@32 accuracy: 13.6% -> 14.2% | Improved |
| add IB advantage to random tree (ablation) | RandTree · GPQA | avg@32 accuracy: 38.1% -> 37.3% | Hurt |
| replace EPTree with IBTree under IB advantage (ablation) | IBTPO · AMC 24 | avg@32 accuracy: 42.3% -> 46.0% | Improved |
| use IB-Score guided branching instead of entropy guided (ablation) | tree sampling G=12 · DAPO-Math-17K subset | Avg-Rate: 21.6% -> 23.2% | Improved |
| use random branching instead of independent sampling (ablation) | tree sampling G=12 · DAPO-Math-17K subset | Eff-Rate: 59.8% -> 48.4% | Hurt |
| vary beta from 1.0 to 5.0 in IB-Score branching (ablation) | IBTree · DAPO-Math-17K subset | Eff-Rate: 60.0% -> 60.2% | Improved |
| add local IB advantage with lambda=0.1 to global advantage (ablation) | IBTPO · AMC 24 | avg@32 accuracy: 43.8% -> 46.0% | Improved |
Closest papers by meaning
- DeepSearch: Overcome the Bottleneck of Reinforcement Learning with Verifiable Rewards via Monte Carlo Tree Search (ICLR 2026 · similarity 0.88)
- Accelerating RL for LLM Reasoning with Optimal Advantage Regression (NeurIPS 2025 · similarity 0.88)
- One-Way Policy Optimization for Self-Evolving LLMs (ICML 2026 · similarity 0.88)
- Resource-Efficient Reinforcement for Reasoning Large Language Models via Dynamic One-Shot Policy Refinement (ICML 2026 · similarity 0.87)
- On Predictability of Reinforcement Learning Dynamics for Large Language Models (ICLR 2026 · similarity 0.87)
- Agentic Reinforced Policy Optimization (ICLR 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
