Research map / Reinforcement learning / RL from feedback
DEVA: Fine-tuning Multimodal Large Language Models for Visual Perception Tasks
Debasmit Das, Munawar Hayat, Fatih Porikli
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add GFlowNet diversity loss to GRPO (ablation) | GRPO · LISA | avg reward std: 0.234 -> 0.262 | Improved |
| add GFlowNet diversity loss to DAPO (ablation) | DAPO · LISA | avg reward std: 0.201 -> 0.240 | Improved |
| add GFlowNet diversity loss to BNPO (ablation) | BNPO · LISA | avg reward std: 0.193 -> 0.221 | Improved |
| add diversity loss to ViRFT (ablation) | ViRFT (GRPO) · LISA test | mIoU (Qwen2-VL-2B): 37.6 -> 43.6 | Improved |
| add global entropic exploration regularization (ablation) | ViRFT + diversity loss · LISA test | mIoU (Qwen2-VL-2B): 43.6 -> 44.7 | Improved |
| add alignment volume reward (ablation) | ViRFT + diversity + exploration · LISA test | mIoU (Qwen2-VL-2B): 44.7 -> 46.7 | Improved |
| add harmonic reward aggregation (ablation) | ViRFT + diversity + exploration + volume · LISA test | mIoU (Qwen2-VL-2B): 46.7 -> 48.9 | Improved |
| add all DEVA components to ViRFT | ViRFT · COCO 4-shot detection | mAP (Qwen2-VL-2B): 40.6 -> 47.3 | Improved |
| add all DEVA components to ViRFT | ViRFT · COCO 4-shot detection | mAP (Qwen2-VL-7B): 54.3 -> 60.0 | Improved |
| add diversity loss to ViRFT (ablation) | ViRFT · COCO 4-shot detection | mAP (Qwen2-VL-2B): 40.6 -> 43.9 | Improved |
| use CoT-style data in SFT (ablation) | SFT · COCO 4-shot detection | mAP (Qwen2-VL-2B): 25.2 -> 29.7 | Improved |
| replace pairwise alignment reward with alignment volume reward (ablation) | DEVA · LISA test | mIoU | Improved |
Closest papers by meaning
- CoVFT: Context-aware Visual Fine-tuning for Multimodal Large Language Models (CVPR 2026 · similarity 0.91)
- What Can RL Bring to VLA Generalization? An Empirical Study (NeurIPS 2025 · similarity 0.91)
- Reason-RFT: Reinforcement Fine-Tuning for Visual Reasoning of Vision Language Models (NeurIPS 2025 · similarity 0.90)
- Perception-Aware Policy Optimization for Multimodal Reasoning (ICLR 2026 · similarity 0.90)
- Perception-R1: Advancing Multimodal Reasoning Capabilities of MLLMs via Visual Perception Reward (ICLR 2026 · similarity 0.90)
- On Robustness and Chain-of-Thought Consistency of RL-Finetuned VLMs (ICML 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
