Research map / Reinforcement learning / RL from feedback
Learning to Explore: Scaling Agentic Reasoning via Exploration-Aware Policy Optimization
Xingyuan Hua, Sheng Yue, Ju Ren
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| naive test-time exploration added to GRPO | GRPO Qwen-VL-2B · AndroidWorld | success rate: 52.3 -> 55.1 | Improved |
| naive test-time exploration added to GRPO | GRPO Qwen-VL-8B · AndroidWorld | success rate: 55.4 -> 52.9 | Hurt |
| naive test-time exploration added to DAPO | DAPO Qwen-VL-2B · AndroidWorld | success rate: 56.8 -> 54.6 | Hurt |
| naive test-time exploration added to GiGPO | GiGPO Qwen-VL-2B · AndroidWorld | success rate: 56.0 -> 53.1 | Hurt |
| naive test-time exploration added to LAMER | LAMER Qwen-VL-2B · AndroidWorld | success rate: 61.2 -> 56.3 | Hurt |
| naive test-time exploration added to LAMER | LAMER Qwen-VL-4B · AndroidWorld | success rate: 61.5 -> 58.2 | Hurt |
| naive test-time exploration added to LAMER | LAMER Qwen-VL-8B · AndroidWorld | success rate: 61.0 -> 57.3 | Hurt |
| add exploratory reward (ablation) | Ours · all four environments | success rate | Improved |
| add exploration-aware grouping (ablation) | Ours · all four environments | success rate | Improved |
| add format reward (ablation) | Ours · all four environments | success rate | Improved |
| train on AndroidWorld and test on unseen OSWorld | Ours 8B · OSWorld overall | success rate: 60.64 | Improved |
| increase discount factor gamma toward 1.0 (ablation) | Ours · text and GUI environments | success rate | Mixed |
Closest papers by meaning
- LLM-Explorer: A Plug-in Reinforcement Learning Policy Exploration Enhancement Driven by Large Language Models (NeurIPS 2025 · similarity 0.91)
- Expanding LLM Agent Boundaries with Strategy-Guided Exploration (NeurIPS 2026 · similarity 0.90)
- Decoupling Exploration and Policy Optimization: Uncertainty Guided Tree Search for Hard Exploration (NeurIPS 2026 · similarity 0.90)
- Learning Agentic Policy from Action Guidance (NeurIPS 2026 · similarity 0.89)
- Imagine, Initialize, and Explore: An Effective Exploration Method in Multi-Agent Reinforcement Learning (AAAI 2024 · similarity 0.89)
- DISCOVER: Automated Curricula for Sparse-Reward Reinforcement Learning (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
