Research map / Reinforcement learning / RL from feedback
T$^2$PO: Uncertainty-Guided Exploration Control for Stable Multi-Turn Agentic Reinforcement Learning
Haixin Wang, Hejie Cui, Chenwei Zhang, Xin Liu, Shuowei Jin, Shijie Geng, Xinyang Zhang, Nasser Zalmout, Zhenyu Shi, Yizhou Sun
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add rejective fine-tuning cold start (ablation) | T2PO · WebShop | success rate: 61.32 -> 81.64 | Improved |
| add token-level thinking intervention (ablation) | T2PO · WebShop | success rate: 73.27 -> 81.64 | Improved |
| add turn-level dynamical sampling (ablation) | T2PO · WebShop | task score: 72.40 -> 93.84 | Improved |
| add turn-level dynamical sampling (ablation) | T2PO · WebShop | success rate: 63.67 -> 81.64 | Improved |
| use TTI+TDS instead of a lengthy reward (ablation) | T2PO · WebShop | success rate: 65.87 -> 81.64 | Improved |
| use TTI+TDS instead of short CoT cold-start (ablation) | T2PO · WebShop | success rate: 71.29 -> 81.64 | Improved |
| use TTI+TDS instead of void turn filtering (ablation) | T2PO · WebShop | success rate: 76.20 -> 81.64 | Improved |
| use TTI+TDS instead of a hard thinking budget (ablation) | T2PO · WebShop | success rate: 79.21 -> 81.64 | Improved |
| use T2PO instead of GiGPO on Qwen3-4B-RFT | GiGPO · ALFWorld | success rate: 80.47 -> 90.23 | Improved |
| use T2PO instead of GiGPO on Qwen3-8B-RFT | GiGPO · WebShop | success rate: 77.92 -> 82.42 | Improved |
| use T2PO instead of GiGPO on search QA (Qwen3-4B) | GiGPO · Search QA average | average score: 52.97 -> 54.93 | Improved |
| increase rollout batch size from 8 to 16 (group size 8) (ablation) | T2PO · WebShop | success rate: 80.97 -> 81.64 | No clear effect |
Closest papers by meaning
- AEM: Adaptive Entropy Modulation for Multi-Turn Agentic Reinforcement Learning (NeurIPS 2026 · similarity 0.88)
- ReLaX: Reasoning with Latent Exploration for Large Reasoning Models (CVPR 2026 · similarity 0.88)
- Decoupling Exploration and Policy Optimization: Uncertainty Guided Tree Search for Hard Exploration (NeurIPS 2026 · similarity 0.88)
- STARE: Surprisal-Guided Token-Level Advantage Reweighting for Policy Entropy Stability (NeurIPS 2026 · similarity 0.88)
- Efficient Model-Based Reinforcement Learning Through Optimistic Thompson Sampling (ICLR 2025 · similarity 0.87)
- LLM-Explorer: A Plug-in Reinforcement Learning Policy Exploration Enhancement Driven by Large Language Models (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
