Research map / Reinforcement learning / RL from feedback
Controllable Exploration in Hybrid-Policy RLVR for Multi-Modal Reasoning
Zhuoxu Huang, Mengxi Jia, Hao Sun, Xuelong Li, Jungong Han
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add CalibRL controllable exploration on top of GRPO | GRPO Qwen2.5-VL-7B · geometry avg (GeoEval, Geo3K, GeoQA) | accuracy (%): 39.48 -> 44.93 | Improved |
| add CalibRL to GRPO on out-of-domain benchmarks | GRPO Qwen2.5-VL-7B · out-of-domain avg (7 benchmarks) | accuracy (%): 57.24 -> 59.36 | Improved |
| add LUFFY hybrid-policy expert guidance to GRPO | GRPO Qwen2.5-VL-7B · geometry avg | accuracy (%): 39.48 -> 38.64 | Hurt |
| add RL-PLUS to GRPO | GRPO Qwen2.5-VL-7B · geometry avg | accuracy (%): 39.48 -> 34.68 | Hurt |
| SFT then GRPO instead of GRPO alone | Qwen2.5-VL-7B · GeoEval | accuracy (%): 26.15 -> 6.00 | Hurt |
| use DAPO higher clipping instead of GRPO | GRPO Qwen2.5-VL-7B · geometry avg | accuracy (%): 39.48 -> 39.55 | No clear effect |
| CalibRL vs GRPO on smaller model | Qwen2.5-VL-3B · 5-benchmark avg | accuracy (%): 34.44 -> 37.09 | Improved |
| CalibRL vs GRPO on InternVL3-8B | InternVL3-8B · 5-benchmark avg | accuracy (%): 45.37 -> 47.42 | Improved |
| add advantage weighting |A_i| to controllable exploration (ablation) | CalibRL · 8-benchmark avg | accuracy (%): 45.30 -> 50.59 | Improved |
| use LeakyReLU slope alpha 0.5 instead of 0.3 or 0.8 (ablation) | CalibRL · 8-benchmark avg | accuracy (%): 49.91 / 49.85 -> 50.59 | Improved |
| increase balance weight lambda from 0.1 to 0.3 or 0.5 (ablation) | CalibRL · 8-benchmark avg | accuracy (%): 50.59 -> 44.58 / 45.13 | Hurt |
| add entropy coefficient 0.01 to GRPO (ablation) | GRPO · geometry avg | accuracy (%): 39.48 -> 36.79 | Hurt |
Closest papers by meaning
- Beyond Pass@ 1: Self-Play with Variational Problem Synthesis Sustains RLVR (ICLR 2026 · similarity 0.92)
- Selective Expert Guidance for Effective and Diverse Exploration in Reinforcement Learning of LLMs (ICLR 2026 · similarity 0.91)
- The Reasoning Boundary Paradox: How Reinforcement Learning Constrains Language Models (NeurIPS 2026 · similarity 0.91)
- Resource-Efficient Reinforcement for Reasoning Large Language Models via Dynamic One-Shot Policy Refinement (ICML 2026 · similarity 0.91)
- SAGE: Shaping Anchors for Guided Exploration in RLVR of LLMs (ICML 2026 · similarity 0.91)
- Exploration vs Exploitation: Rethinking RLVR through Clipping, Entropy, and Spurious Reward (ICLR 2026 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
