Research map / Vision-language and multimodal / Multimodal LLMs
VisRL: Intention-Driven Visual Perception via Reinforced Reasoning
Zhangquan Chen, Xufang Luo, Dongsheng Li
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add RL1 (bounding box DPO) after SFT (ablation) | VisRL on LLaVA-1.5-7B · Visual CoT benchmark (11 datasets, average) | average accuracy: 0.538 -> 0.568 | Improved |
| add RL2 (joint box and response DPO) after RL1 (ablation) | VisRL on LLaVA-1.5-7B · Visual CoT benchmark (11 datasets, average) | average accuracy: 0.568 -> 0.605 | Improved |
| add RL1 and RL2 to SFT on Qwen2.5-VL-7B (ablation) | VisRL · Visual CoT benchmark (11 datasets, average) | average accuracy: 0.730 -> 0.796 | Improved |
| add RL1 before RL2 (instead of RL2-only) (ablation) | VisRL on Qwen2.5-VL-7B · Visual CoT benchmark, final-response training strategies (average) | average accuracy: 0.759 -> 0.796 | Improved |
| use bounding box CoT RL instead of direct response DPO (ablation) | VisRL on Qwen2.5-VL-7B · Visual CoT benchmark, final-response training strategies (average) | average accuracy: 0.656 -> 0.796 | Improved |
| use bounding box CoT RL instead of PPO on final response (ablation) | VisRL on Qwen2.5-VL-7B · Visual CoT benchmark, final-response training strategies (average) | average accuracy: 0.661 -> 0.796 | Improved |
| SFT 30k instead of 438k bounding box data (ablation) | LLaVA-1.5-7B · Visual CoT benchmark (11 datasets, average) | average accuracy: 0.547 -> 0.538 | No clear effect |
| add RL after SFT for bounding box accuracy (ablation) | VisRL LLaVA-1.5-7B · Visual CoT benchmark bounding box detection (Acc@0.5, average, LLaVA-1.5-7B) | Top-1 Accuracy@0.5: 0.374 -> 0.437 | Improved |
| one iteration of data regeneration and retraining | VisRL on LLaVA-1.5 · MME | score: 1526.3 -> 1560.0 | Improved |
| add diversity controller to data generation (ablation) | VisRL data pipeline · Data generation pipeline (Qwen2.5-VL-7B) | WP-LN proportion: 12.62% reduction without it | Improved |
| add bounding box evaluation to data generation (ablation) | VisRL data pipeline · Data generation pipeline (Qwen2.5-VL-7B) | WP-LN proportion: 33.62% drop without it | Improved |
| use pre-SFT model as critic instead of SFTed model (ablation) | VisRL data pipeline · Data generation pipeline (Qwen2.5-VL-7B) | WP-LN proportion: 54.68% -> 64.64% | Improved |
Closest papers by meaning
- More Thought, Less Accuracy? On the Dual Nature of Reasoning in Vision-Language Models (ICLR 2026 · similarity 0.91)
- DeepEyes: Incentivizing "Thinking with Images" via Reinforcement Learning (ICLR 2026 · similarity 0.90)
- Activating Visual Context and Commonsense Reasoning Through Masked Prediction in VLMs (AAAI 2026 · similarity 0.90)
- CoFFT: Chain of Foresight-Focus Thought for Visual Language Models (NeurIPS 2025 · similarity 0.90)
- PeRL: Permutation-Enhanced Reinforcement Learning for Interleaved Vision-Language Reasoning (NeurIPS 2025 · similarity 0.89)
- Visual Planning: Let's Think Only with Images (ICLR 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
