Research map / Reinforcement learning / RL from feedback
A Rolling Stone Gathers No Moss: Adaptive Policy Optimization for Stable Self-Evaluation in Large Multimodal Models
Wenkai Wang, Hongcan Guo, Zheqi Lv, Shengyu Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Adaptive Reward Model (ablation) | AdaPO (Qwen2.5-VL-7B) · 8 benchmarks avg (Qwen2.5-VL-7B) | Avg acc@t2: 53.8 -> 60.2 | Improved |
| add Adaptive Reward Model (ablation) | AdaPO (Qwen2.5-VL-7B) · 8 benchmarks avg (Qwen2.5-VL-7B) | Avg acc@t1: 55.1 -> 57.6 | Improved |
| add Reward Aware KL (ablation) | AdaPO (Qwen2.5-VL-7B) · 8 benchmarks avg (Qwen2.5-VL-7B) | Avg acc@t1: 56.2 -> 57.6 | Improved |
| add Reward Aware KL (ablation) | AdaPO (Qwen2.5-VL-7B) · 8 benchmarks avg (Qwen2.5-VL-7B) | Avg acc@t2: 58.6 -> 60.2 | Improved |
| add hybrid data filtering (ablation) | AdaPO (Qwen2.5-VL-7B) · 8 benchmarks avg (Qwen2.5-VL-7B) | Avg acc@t2: 57.5 -> 60.2 | Improved |
| add hybrid data filtering (ablation) | AdaPO (Qwen2.5-VL-7B) · 8 benchmarks avg (Qwen2.5-VL-7B) | Avg acc@t1: 54.3 -> 57.6 | Improved |
| prompt base model for self-evaluation (no training) | Qwen2.5-VL-7B · MathVista | acc@t2 vs acc@t1: 68.2 -> 60.0 | Hurt |
| fixed-reward GRPO for self-evaluation | Qwen2.5-VL-7B · AI2D | acc@t2 vs acc@t1: 80.5 -> 80.9 | No clear effect |
| replace GRPO with AdaPO | Qwen2.5-VL-7B · AI2D | acc@t2: 80.9 -> 85.5 | Improved |
| replace SCoRe staged RL with AdaPO | Qwen2.5-VL-7B · MMMU | acc@t1: 39.4 -> 54.4 | Improved |
| replace GRPO with AdaPO | Qwen2.5-VL-7B · WeMath | acc@t2: 65.7 -> 73.4 | Improved |
| apply SFT or STaR for self-evaluation | Qwen2.5-VL-7B · WeMath | acc@t2 vs acc@t1: 61.9 -> 51.1 | Hurt |
Closest papers by meaning
- Diving into Self-Evolving Training for Multimodal Reasoning (ICML 2025 · similarity 0.89)
- SRPO: Enhancing Multimodal LLM Reasoning via Reflection-Aware Reinforcement Learning (NeurIPS 2025 · similarity 0.88)
- Enhancing the Outcome Reward-based RL Training of MLLMs with Self-Consistency Sampling (NeurIPS 2025 · similarity 0.88)
- Training Language Models to Self-Correct via Reinforcement Learning (ICLR 2025 · similarity 0.88)
- MSRL: Scaling Generative Multimodal Reward Modeling via Multi-Stage Reinforcement Learning (CVPR 2026 · similarity 0.87)
- BOTS: A Unified Framework for Bayesian Online Task Selection in LLM Reinforcement Finetuning (ICLR 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
