Research map / Reinforcement learning / RL from feedback
From Exploration to Exploitation: A Two-Stage Entropy RLVR Approach for Noise-Tolerant MLLM Training
Donglai Xu, Hongzheng Yang, Yuzhi Zhao, Pingping Zhang, Jinpeng Chen, Wenao Ma, Zhijian Hou, Mengyang Wu, Xiaolei Li, Senkang Hu, Ziyi Guan, Jason Chun Lok Li and 1 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add two-stage entropy schedule (maximize then minimize) to GRPO | GRPO, Qwen2.5-VL-3B · ScreenSpot GUI grounding, 50% label noise | accuracy (%): 76.2 -> 80.2 | Improved |
| add two-stage entropy schedule to GRPO | GRPO, Qwen2.5-VL-3B · ScreenSpot GUI grounding, 100% label noise | accuracy (%): 71.0 -> 75.8 | Improved |
| add two-stage entropy schedule to GRPO | GRPO, Qwen2.5-VL-3B · ScreenSpot GUI grounding, 0% label noise | accuracy (%): 82.2 -> 83.6 | Improved |
| add constant entropy minimization to GRPO (ablation) | GRPO, Qwen2.5-VL-3B · ScreenSpot GUI grounding, 0% label noise | accuracy (%): 82.2 -> 79.0 | Hurt |
| add constant entropy minimization to GRPO (ablation) | GRPO, Qwen2.5-VL-3B · ScreenSpot GUI grounding, 100% label noise | accuracy (%): 71.0 -> 73.2 | Improved |
| add constant entropy maximization to GRPO (ablation) | GRPO, Qwen2.5-VL-3B · ScreenSpot GUI grounding, 50% label noise | accuracy (%): 76.2 -> 77.8 | Improved |
| add two-stage entropy schedule to GRPO | GRPO, Qwen2.5-VL-3B · Pets37 4-shot, 50% label noise | accuracy (%): 68.5 -> 68.4 | No clear effect |
| add two-stage entropy schedule to GRPO | GRPO, Qwen2.5-VL-3B · Pets37 4-shot, 100% label noise | accuracy (%): 54.7 -> 54.3 | No clear effect |
| add two-stage entropy schedule to GRPO | GRPO, Qwen2-VL-7B · ScreenSpot GUI grounding, 50% label noise | accuracy (%): 61.2 -> 69.8 | Improved |
| add two-stage entropy schedule to GRPO | GRPO, Qwen2-VL-2B · ScreenSpot GUI grounding, 50% label noise | accuracy (%): 32.8 -> 25.2 | Hurt |
| add two-stage entropy schedule to GRPO | GRPO, Qwen2-VL-2B · OVOD COCO unseen, 50% label noise | mAP@0.5: 15.94 -> 19.47 | Improved |
| order entropy phases as max then min instead of min then max (ablation) | two-stage GRPO, Qwen2.5-VL-3B · ScreenSpot GUI grounding, 100% label noise | accuracy (%): 70.2 -> 75.8 | Improved |
Closest papers by meaning
- You Only Need Minimal RLVR Training: Extrapolating LLMs via Rank-1 Trajectories (NeurIPS 2026 · similarity 0.90)
- Learning to Explore with Parameter-Space Noise: A Deep Dive into Parameter-Space Noise for Reinforcement Learning with Verifiable Rewards (NeurIPS 2026 · similarity 0.90)
- Beyond Pass@ 1: Self-Play with Variational Problem Synthesis Sustains RLVR (ICLR 2026 · similarity 0.89)
- Listwise Policy Optimization: Group-based RLVR as Target-Projection on the LLM Response Simplex (NeurIPS 2026 · similarity 0.89)
- Exploration vs Exploitation: Rethinking RLVR through Clipping, Entropy, and Spurious Reward (ICLR 2026 · similarity 0.89)
- Reinforcement Learning for Reasoning in Large Language Models with One Training Example (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
