Research map / Vision-language and multimodal / Multimodal LLMs

Video Spatial Reasoning with Object-Centric 3D Rollout

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Spatial reasoning in VLMs · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Object-Centric 3D Rollout (OCR), a GRPO variant that adds Gaussian-noised object regions to videos as parallel rollouts, and trains a 3B Qwen2.5-VL on new OCR-SFT/OCR-RL data to reach 47.5% on VSI-Bench.

Problem

MLLMs show query-locked reasoning in video spatial reasoning, attending narrowly to objects mentioned in the question and ignoring contextual objects, and existing GRPO-based methods overfit on scarce 3D indoor video data.

Approach

Perturb selected objects' projected 3D regions in video frames with Gaussian noise, generate parallel rollouts on both vanilla and region-noisy videos, compute advantages over the combined reward group, update gradients only from clean rollouts, and anneal the number of perturbed objects over training.

Key results

• The 3B OCR model reaches 47.5% average on VSI-Bench, above 7B models Video-R1 (37.1), vsGRPO-V (40.7) and SpaceR (SG-RLVR, 45.6).
• Ablation on rollout policy: OCR 47.5 vs. vanilla GRPO 43.9, T-GRPO 44.1, NoisyRollout 45.6, downsample rollout 44.2 on VSI-Bench average.
• Linear object-count annealing (50% to 0%) gives 47.5 vs. fixed 25% at 45.0, exponential 46.0, cosine 46.3.

Surprising findings

• A fixed 25% object perturbation schedule performs worse than the annealed schedule, suggesting the schedule matters.

Limitations

• Relies on GPT-4o for CoT annotation and filtering, so data quality depends on a proprietary model.
• Only evaluated on VSI-Bench; generalization to other spatial benchmarks is not shown.
• Perturbation uses 3D bounding boxes and projection, which requires ground-truth 3D annotations for training scenes.

Experimental setup

Datasets: OCR-SFT OCR-RL VSI-Bench SpaceR annotations VLM-3R annotations ScanNet ScanNet++ ARKitScenes

Benchmarks: VSI-Bench

Metrics: accuracy (average over 8 tasks)

Baselines: Video-R1 vsGRPO-V SpaceR VG-LLM T-GRPO NoisyRollout Vanilla GRPO GPT-4o Gemini-1.5-Pro

Builds on: GRPO Video-R1 T-GRPO NoisyRollout VSI-Bench

Closest papers by meaning

• From Flatland to Space: Teaching Vision-Language Models to Perceive and Reason in 3D (NeurIPS 2025 · similarity 0.91)
• Spatial Understanding from Videos: Structured Prompts Meet Simulation Data (NeurIPS 2025 · similarity 0.91)
• Spatial-MLLM: Boosting MLLM Capabilities in Visual-based Spatial Intelligence (NeurIPS 2025 · similarity 0.90)
• VLM-3R: Vision-Language Models Augmented with Instruction-Aligned 3D Reconstruction (CVPR 2026 · similarity 0.90)
• Pursuing Minimal Sufficiency in Spatial Reasoning (ICLR 2026 · similarity 0.90)
• SpatialReasoner: Towards Explicit and Generalizable 3D Spatial Reasoning (NeurIPS 2025 · similarity 0.90)