Research map / Vision-language and multimodal / Multimodal LLMs
Fine-Grained Preference Optimization Improves Spatial Reasoning in VLMs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SpatialReasoner-R1, a VLM for LongCoT spatial reasoning trained with a segment-wise fine-grained DPO (fDPO) on preference pairs generated by a multi-model MCTS (M3CTS) and depth-guided rewards.
Problem
Vision-language models struggle with fine-grained, multi-step spatial reasoning, and standard DPO treats the whole reasoning trace as a single unit despite descriptive and reasoning segments needing different optimization.
Approach
fDPO splits each response into description and reasoning segments, computes segment-specific score margins from fine-grained rewards, and adaptively scales the DPO temperature per segment; M3CTS uses several VLMs within an MCTS loop to generate diverse LongCoT trajectories scored by multi-model evaluators.
Key results
- fDPO 8B improves over DPO 8B by 4.1% on spatial qualitative and 9.0% on quantitative tasks (relative).
- SpatialReasoner-R1 fDPO 8B reaches 95.59% qualitative accuracy on SpatialRGPT-Bench, versus 92.69% for SpatialRGPT-8B.
- fDPO 8B gets 77.30% quantitative accuracy vs 61.42% for SpatialRGPT-8B, and reports 89.71 POPE and 51.10 HallusionBench on general benchmarks.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace standard DPO with fDPO (ablation) | SpatialReasoner-R1 8B · SpatialRGPT-Bench qualitative | Qual. Acc.: 91.48 -> 95.59; 4.1% | Helped |
| replace standard DPO with fDPO (ablation) | SpatialReasoner-R1 8B · SpatialRGPT-Bench quantitative | Quan. Acc.: 68.22 -> 77.30; 9.0% | Helped |
| train with fDPO instead of SpatialRGPT-8B | SpatialRGPT-Bench · qualitative | Qual. Acc.: 92.69 -> 95.59 | Helped |
| train with fDPO instead of SpatialRGPT-8B | SpatialRGPT-Bench · quantitative | Quan. Acc.: 61.42 -> 77.30 | Helped |
| set alpha to 30% rather than 10% (ablation) | fDPO · SpatialRGPT-Bench quantitative | Direct Dist.: 53.38 -> 60.13 | Helped |
| set alpha to 30% rather than 20% (ablation) | fDPO · SpatialRGPT-Bench quantitative | Direct Dist.: 56.76 -> 60.13 | Helped |
| increase alpha from 30% to 40% (ablation) | fDPO · SpatialRGPT-Bench quantitative | Direct Dist.: 60.13 -> 58.11 | Hurt |
| set lambda to 0.6 rather than 0.2 (ablation) | fDPO at alpha 30% · SpatialRGPT-Bench quantitative | Direct Dist.: 54.05 -> 60.13 | Helped |
| set lambda to 0.6 rather than 0.4 (ablation) | fDPO at alpha 30% · SpatialRGPT-Bench quantitative | Direct Dist.: 57.38 -> 60.13 | Helped |
| increase lambda from 0.6 to 0.8 (ablation) | fDPO at alpha 30% · SpatialRGPT-Bench quantitative | Direct Dist.: 60.13 -> 59.45 | Hurt |
| train with fDPO rather than SpatialRGPT-8B | general vision-language benchmarks · POPE | score: 85.50 -> 89.71 | Helped |
| train with fDPO rather than SpatialRGPT-8B | general vision-language benchmarks · SEED-Bench | score: 67.00 -> 76.21 | Helped |
Surprising findings
- The 4B fDPO model outperforms the much larger InternVL2.5-78B on spatial benchmarks.
- Ablation shows that overly high alpha (40%) degrades performance, indicating instability from aggressive segment reweighting.
Limitations
- Relies on explicit region/visual prompt inputs to disambiguate object references rather than grounding from language alone.
- Limited to 2D spatial reasoning; 3D and embodied extensions left for future work.
- Failure cases show over-reliance on default object-size priors in vertical estimation.
Experimental setup
Datasets: Open Spatial Open Spatial Reasoning (constructed) SpatialRGPT-Bench MME POPE SEED-Bench AI2D SQA-test MMMU MMStar HallusionBench
Benchmarks: SpatialRGPT-Bench MME POPE SEED-Bench AI2D SQA-test MMMU MMStar HallusionBench
Metrics: success rate within 25% (quantitative) accuracy (qualitative) accuracy
Baselines: Gemini 2.0 Flash Llama 4 Maverick Gemini 1.5 Pro ChatGPT-4o SpatialBot-3B SpaceThinker Qwen2.5VL-3B InternVL2.5-78B Sa2VA SpatialRGPT-8B
Builds on: DPO Sa2VA DeepSeek-R1-Zero Monte Carlo Tree Search
Closest papers by meaning
- SpatialRGPT: Grounded Spatial Reasoning in Vision-Language Models (NeurIPS 2024 · similarity 0.92)
- SpatialLadder: Progressive Training for Spatial Reasoning in Vision-Language Models (ICLR 2026 · similarity 0.91)
- Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Models (NeurIPS 2024 · similarity 0.90)
- SpatiaLQA: A Benchmark for Evaluating Spatial Logical Reasoning in Vision-Language Models (CVPR 2026 · similarity 0.90)
- Expand VSR Benchmark for VLLM to Expertize in Spatial Rules (AAAI 2025 · similarity 0.90)
- Spatial-DISE: A Unified Benchmark for Evaluating Spatial Reasoning in Vision-Language Models (ICLR 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.