Research map / Vision-language and multimodal / Multimodal LLMs

STAR-R1: Multi-View Spatial TrAnsformation Reasoning by Reinforcing Multimodal LLMs

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes STAR-R1, a two-stage SFT-then-RL framework with referential-selection and answer rewards that improves Qwen2.5-VL-7B on multi-view spatial transformation reasoning benchmarks.

Problem

MLLMs perform poorly on multi-view spatial reasoning, requiring cross-view object correspondence; SFT fails to capture cross-view consistency and vanilla RL often misses key referential objects.

Approach

Combines process-supervised SFT cold start (Gemini-2.5-Pro-generated structured CoTs: per-view referential analysis, cross-view mapping, answer inference) with GRPO using dense per-transformation rewards with penalties plus a referential-selection reward.

Key results

• STAR-R1 (7B) reaches 61.4 TAcc on TVR versus 48.7 for STAR-SFT and 3.8 for base Qwen2.5-VL-7B.
• STAR-R1 scores 31.4 on MMSI-Bench and 98.5 on MindCube-Tiny Rotation, exceeding prior open-source SOTA by 45.5 points on Rotation.
• On OOD TVR split, RL reaches 53.9 TAcc versus 30.9 for SFT, a 23-point gain.

Surprising findings

• SFT achieves higher in-domain TVR accuracy (84.2 TAcc) than RL (76.3), yet RL generalizes better out-of-domain.
• Removing the penalty term causes reward hacking where the model enumerates possible triplets.

Limitations

• Relies on Gemini-2.5-Pro-generated CoTs for cold-start data.
• Gains on some benchmarks (e.g., MMSI-Bench, SPAR-Bench) are modest relative to closed-source models; MMSI-Bench Acc remains 31.4.

Experimental setup

Datasets: MindCube SPAR-7M TRANCE

Benchmarks: TVR MMSI-Bench MindCube-Tiny SPAR-Bench

Metrics: TAcc Diff NDiff Accuracy Object Total Accuracy

Baselines: GPT-4o o3 Claude-4-Sonnet Gemini-2.0-flash Qwen2.5-VL-7B Qwen2.5-VL-32B InternVL3 STAR-SFT STAR-Zero-RL

Builds on: GRPO DeepSeek-R1 TRANCE

Closest papers by meaning

• From Correspondence to Actions: Human-Like Multi-Image Spatial Reasoning in Multi-modal Large Language Models (ICML 2026 · similarity 0.93)
• Seeing from Another Perspective: Evaluating Multi-View Understanding in MLLMs (AAAI 2026 · similarity 0.91)
• Actial: Activate Spatial Reasoning Ability of Multimodal Large Language Models (NeurIPS 2025 · similarity 0.91)
• Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Models (NeurIPS 2024 · similarity 0.91)
• Coarse Correspondences Boost Spatial-Temporal Reasoning in Multimodal Language Model (CVPR 2025 · similarity 0.91)
• Multi-SpatialMLLM: Multi-Frame Spatial Understanding with Multi-Modal Large Language Models (CVPR 2026 · similarity 0.90)