Research map / Vision-language and multimodal / Multimodal LLMs

From Correspondence to Actions: Human-Like Multi-Image Spatial Reasoning in Multi-modal Large Language Models

ICML 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Spatial reasoning in VLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes HATCH, a two-stage training framework (PaStA patch-level cross-view correspondence alignment, then ActoR action-then-answer GRPO with geometric action rewards) that improves multi-image spatial reasoning in Qwen2.5-VL-3B.

Problem

Multi-image spatial reasoning, where evidence is spread across multiple viewpoints, remains hard for MLLMs; existing methods incorporate cross-view correspondence and viewpoint transformation only partially and implicitly.

Approach

Explicitly supervise patch-level cross-view correspondence using geometry-derived overlap targets (encoder-only update), then train the model to generate JSON viewpoint-transition actions before answering, optimized with GRPO using geometric action rewards and answer rewards.

Key results

• HATCH reaches 53.6% on SPAR-Bench-MV, +28.7 points over Qwen2.5-VL-3B and matching GPT-5.2 (52.6% on the subset).
• HATCH reaches 50.2% on MindCube-Tiny (+12.4 over base) and 27.0% on MMSI-Bench (+1.4 over base), with 43.6% overall.
• On single-image benchmarks, HATCH improves SPAR-Bench-SI from 23.9% to 45.2% and CV-Bench from 70.7% to 74.5%.

Surprising findings

• Removing the correspondence stage (PaStA) degrades action and QA rewards throughout GRPO training, showing its role as a foundation for action generation.
• Action-based reasoning outperforms free-form natural-language chain-of-thought under the same GRPO setup (53.6 vs 52.8 on SPAR-Bench-MV).

Limitations

• Requires camera intrinsics, poses, and depth maps during training, limiting applicability to in-the-wild data.
• Assumes static scenes; temporal inconsistencies (moving objects) can produce misleading correspondence signals.
• Does not improve over baseline on MindCube Rotation and some MMSI-Bench Motion/MSR categories; large gap to proprietary models on MMSI-Bench.

Experimental setup

Datasets: SPAR-7M SPAR-Bench MindCube MMSI-Bench CV-Bench

Benchmarks: SPAR-Bench-MV MindCube-Tiny MMSI-Bench SPAR-Bench-SI CV-Bench

Metrics: accuracy MRA average accuracy

Baselines: GPT-4.1 GPT-5.2 Gemini-3-Pro InternVL-2.5-4B InternVL-2.5-8B LLaVA-OneVision-4B Qwen2.5-VL-32B Qwen2.5-VL-72B SpaceR-7B Video-R1 Spatial-MLLM-4B SpatialLadder-3B

Builds on: GRPO CroCo chain-of-thought

Closest papers by meaning

• STAR-R1: Multi-View Spatial TrAnsformation Reasoning by Reinforcing Multimodal LLMs (CVPR 2026 · similarity 0.93)
• Seeing from Another Perspective: Evaluating Multi-View Understanding in MLLMs (AAAI 2026 · similarity 0.92)
• Coarse Correspondences Boost Spatial-Temporal Reasoning in Multimodal Language Model (CVPR 2025 · similarity 0.92)
• Actial: Activate Spatial Reasoning Ability of Multimodal Large Language Models (NeurIPS 2025 · similarity 0.92)
• Eliciting Complex Spatial Reasoning in MLLMs through Wide-Baseline Matching (CVPR 2026 · similarity 0.91)
• Is A Picture Worth A Thousand Words? Delving Into Spatial Reasoning for Vision Language Models (NeurIPS 2024 · similarity 0.91)