Research map / Reinforcement learning / RL from feedback
Perception-R1: Pioneering Perception Policy with Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Applies GRPO-based rule-based RL with discriminative rewards (IoU, point distance, edit distance, F1-based detection rewards) to Qwen2-VL-2B and Qwen2.5-VL-3B for perception tasks, finding thinking is unnecessary and RL helps most on high-perplexity tasks like counting and detection.
Problem
Whether rule-based RL (GRPO) with reasoning helps MLLM visual perception tasks such as grounding, OCR, counting, and object detection, and how it compares with SFT.
Approach
Adapts DeepSeek-R1-style GRPO post-training to perception tasks with task-specific discriminative rewards and Hungarian-matching-based multi-subject reward, and argues that perceptual perplexity determines when RL beats SFT and that explicit thinking is unnecessary.
Key results
- Perception-R1 (2B) reaches 81.7 val RefCOCO+ Acc@0.5 and 78.1 Pixmo-Count val vs 60.2 for Qwen2-VL, a +17.9 point gain on counting.
- Perception-R1 built on Qwen2.5-VL-3B achieves 31.9 AP on COCO2017 val versus 16.1 for Qwen2.5-VL-3B, the first pure MLLM reported above 30 AP.
- On ablations, RL-only beats SFT-only and SFT+RL on counting and detection, while SFT is better on grounding and OCR.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply Perception-R1 RL post-training | Qwen2-VL-2B-Instruct · PixMo-Count val | score: 60.2 -> 78.1 | Helped |
| apply Perception-R1 RL post-training | Qwen2.5-VL-3B-Instruct · COCO2017 val | AP: 16.1 -> 31.9 | Helped |
| apply Perception-R1 RL post-training | Qwen2-VL-2B-Instruct · RefCOCO+ val | Acc@0.5: 77.1 -> 81.7 | Helped |
| apply Perception-R1 RL post-training | Qwen2-VL-2B-Instruct · PageOCR en | F1-score: 94.4 -> 98.2 | Helped |
| add bipartite reward matching instead of sequential matching (ablation) | Perception-R1 · PixMo-Count val | score: 77.1 -> 78.1 | Helped |
| add bipartite reward matching instead of sequential matching (ablation) | Perception-R1 · COCO2017 val | AP: 23.5 -> 31.9 | Helped |
| add RL post-training (ablation) | Qwen2-VL-2B-Instruct · PixMo-Count val | score: 60.2 -> 78.1 | Helped |
| add RL post-training (ablation) | Qwen2.5-VL-3B-Instruct · COCO2017 val | AP: 16.1 -> 31.9 | Helped |
| remove explicit thinking process (ablation) | Perception-R1 · COCO2017 val | AP: 25.7 -> 28.1 | Helped |
| remove explicit thinking process (ablation) | Perception-R1 · RefCOCO+ val | Acc@0.5: 67.9 -> 81.7 | Helped |
| use RL only rather than SFT only (ablation) | Qwen2.5-VL-3B-Instruct · COCO2017 val | AP: 25.9 -> 31.9 | Helped |
| use RL only rather than SFT+RL (ablation) | Qwen2.5-VL-3B-Instruct · COCO2017 val | AP: 30.8 -> 31.9 | Helped |
Surprising findings
- Adding an explicit thinking process during RL degraded performance across all four perception tasks (e.g., RefCOCO val 75.1 with thinking vs 89.1 without).
- Several SoTA MLLMs score below 1% on Acc@0.95 for RefCOCO, suggesting current reporting is not discriminative.
Limitations
- Current perception tasks are overly simplistic, limiting RL exploration space.
- Object detection result uses Qwen2.5-VL-3B rather than the 2B main model, and the reward ablation uses only 10k COCO samples.
- Chinese OCR results are zero-shot and no compute or hardware is reported.
Experimental setup
Datasets: RefCOCO RefCOCO+ RefCOCOg PageOCR PixMo-Count COCO2017
Benchmarks: RefCOCO RefCOCO+ RefCOCOg PageOCR PixMo-Count COCO2017 MMBench MMVet MMStar ScienceQA SeedBench MME
Metrics: Acc@0.5 Acc@0.75 Acc@0.95 edit distance F1-score BLEU METEOR counting accuracy AP AP50
Baselines: Qwen2-VL-2B LLaVA-1.5 LLaVA-NeXT LLaVA-OV Qwen2.5-VL-3B GOT YOLOv3 Faster-RCNN DETR SFT
Builds on: GRPO DeepSeek-R1 Kimi-1.5
Closest papers by meaning
- Perception-R1: Advancing Multimodal Reasoning Capabilities of MLLMs via Visual Perception Reward (ICLR 2026 · similarity 0.90)
- Spotlight on Token Perception for Multimodal Reinforcement Learning (ICLR 2026 · similarity 0.89)
- Improving Vision-language Models with Perception-centric Process Reward Models (CVPR 2026 · similarity 0.89)
- PRPO: Perception-Reinforced Policy Optimization via Token-Level Dynamic Advantage Reshaping (NeurIPS 2026 · similarity 0.89)
- All Roads Lead to Rome: Incentivizing Divergent Thinking in Vision-Language Models (CVPR 2026 · similarity 0.88)
- Perception-Aware Policy Optimization for Multimodal Reasoning (ICLR 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.