Research map / Reinforcement learning / RL from feedback
Agentic Reinforcement Learning with Implicit Step Rewards
Xiaoqian Liu, Ke Wang, Yuchuan Wu, Fei Huang, Yongbin Li, Jianbin Jiao, Junge Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add implicit step rewards (iStar) to RLOO | RLOO · WebShop | success rate (%): 77.4 -> 86.5 | Improved |
| add implicit step rewards (iStar) to RLOO | RLOO · VisualSokoban | success rate (%): 86.3 -> 91.7 | Improved |
| add iStar to GRPO on SOTOPIA self-chat hard | GRPO · SOTOPIA self-chat (Hard), Llama3.1-8B | goal completion: 7.92 -> 8.06 | Mixed |
| add iStar to GRPO with GPT-4o partner | GRPO · SOTOPIA GPT-4o partner (Hard), Llama3.1-8B | goal completion: 6.68 -> 7.16 | Improved |
| combine implicit step rewards at advantage level rather than merging into outcome reward (ablation) | iStar · WebShop | success rate (%): 81.3 -> 89.1 | Improved |
| use step-level instead of token-level implicit rewards (ablation) | iStar · WebShop | success rate (%): 82.0 -> 89.1 | Improved |
| use step-level implicit rewards instead of token-level (ablation) | iStar · VisualSokoban | success rate (%): 89.1 -> 93.0 | Improved |
| use implicit step rewards instead of ground-truth step penalties (ablation) | iStar · VisualSokoban | success rate (%): 87.5 -> 93.0 | Improved |
| ground-truth process rewards added to RLOO (ablation) | RLOO · VisualSokoban | success rate (%): 85.9 -> 87.5 | Improved |
| add iStar to smaller base model | RLOO · WebShop, Qwen2.5-1.5B | success rate (%): 71.9 -> 80.5 | Improved |
| add iStar to RLOO (different RL algorithms) | RLOO · WebShop and VisualSokoban | success rate (%): +6.3% on both | Improved |
Closest papers by meaning
- Learning Agentic Policy from Action Guidance (NeurIPS 2026 · similarity 0.90)
- Self-evolving LLM agents with in-distribution Optimization (ICML 2026 · similarity 0.90)
- Agentic Reinforced Policy Optimization (ICLR 2026 · similarity 0.89)
- Supervised Reinforcement Learning: From Expert Trajectories to Step-wise Reasoning (ICLR 2026 · similarity 0.89)
- Latent Reward: LLM-Empowered Credit Assignment in Episodic Reinforcement Learning (AAAI 2025 · similarity 0.89)
- Rewarding Progress: Scaling Automated Process Verifiers for LLM Reasoning (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
