Research map / Vision-language and multimodal / Multimodal LLMs
POLIA: Policy Optimization with Visual-Object-Level Intrinsic Advantage for Multimodal Reasoning
Yiran Zeng, Da Chen, Hangyu Mao, Yuanxing Zhang, Pengfei Wan, Mengchen Zhao
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes POLIA, a group-based RL method that adds visual-object-level intrinsic advantages to address reasoning hallucination in multimodal LLM reasoning.
Problem
Group-based RL methods for LLM text reasoning lack sufficient modeling of multimodal information, causing significant reasoning hallucination in multimodal reasoning.
Approach
Introduces two advantage computation stages over candidate responses that assign intrinsic advantages at the visual-object level, so that policy updates reward grounding in visual objects rather than only final answer correctness.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add visual-object-level intrinsic advantage to GRPO (POLIA vs GRPO, 7B) | Qwen2.5-VL-7B · VSR | ACC (%): +22.3% | Improved |
| add visual-object-level intrinsic advantage to GRPO (7B) | Qwen2.5-VL-7B · TallyQA | ACC (%): +8.7% | Improved |
| add visual-object-level intrinsic advantage to GRPO (7B) | Qwen2.5-VL-7B · GQA | ACC (%): +11.3% | Improved |
| add visual-object-level intrinsic advantage to GRPO (7B) | Qwen2.5-VL-7B · MathVista | ACC (%): +9.3% | Improved |
| add intrinsic advantage to RL baselines (3B) | Qwen2.5-VL-3B · VSR | ACC (%): +18.4% | Improved |
| add intrinsic advantage to RL baselines (3B) | Qwen2.5-VL-3B · TallyQA | ACC (%): +7.2% | Improved |
| add answer-level extrinsic advantage (ablation) | POLIA · VSR and TallyQA training | normalized training reward | Improved |
| add object-level intrinsic advantage (ablation) | POLIA · VSR and TallyQA training | normalized training reward | Improved |
| add intrinsic advantage to reduce visual object group count (ablation) | POLIA · TallyQA (N=8) | number of visual object groups: increases by 50% | Improved |
| increase generation size from 8 to 16 or 24 (ablation) | POLIA · TallyQA | ACC | Improved |
| increase generation size from 16 to 24 (ablation) | POLIA · TallyQA | ACC | No clear effect |
| add intrinsic advantage computation overhead | POLIA-3B · per-iteration training time | seconds: 0.002 s | No clear effect |
Experimental setup
Builds on: GRPO
Closest papers by meaning
- Perception-Aware Policy Optimization for Multimodal Reasoning (ICLR 2026 · similarity 0.89)
- Stable and Efficient Single-Rollout RL for Multimodal Reasoning (CVPR 2026 · similarity 0.88)
- SRPO: Enhancing Multimodal LLM Reasoning via Reflection-Aware Reinforcement Learning (NeurIPS 2025 · similarity 0.88)
- Enhancing Multi-Modal LLMs Reasoning via Difficulty-Aware Group Normalization (ICML 2026 · similarity 0.88)
- Perception-R1: Advancing Multimodal Reasoning Capabilities of MLLMs via Visual Perception Reward (ICLR 2026 · similarity 0.88)
- Critic-V: VLM Critics Help Catch VLM Errors in Multimodal Reasoning (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
