Research map / Vision-language and multimodal / Multimodal LLMs

ConsisVLA-4D: Advancing Spatiotemporal Consistency in Efficient 3D-Perception and 4D-Reasoning for Robotic Manipulation

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Spatial reasoning in VLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ConsisVLA-4D, an OpenVLA-based VLA that adds VGGT/DINOv2 3D geometry from RGB-only views and trains a training-only 4D reasoner (dynamic objects and global depth) for efficient spatiotemporally consistent action prediction.

Problem

VLA models struggle with 3D spatial understanding and 4D spatiotemporal reasoning; existing 3D methods need extra sensors and add heavy computation.

Approach

Fuse SigLIP semantic tokens (instruction-filtered top-K) with VGGT 3D features and DINOv2 geometric features into compact tokens, then use a training-only CS-Thinker that predicts future dynamic object tokens and global depth tokens as auxiliary supervision, so inference only needs the compact tokens.

Key results

• On LIBERO, ConsisVLA-4D averages 98.1% success versus 97.1% for OpenVLA-OFT and 76.5% for OpenVLA.
• Inference latency 0.110 s (72.7 Hz) vs 0.137 s for OpenVLA-OFT in simulation; FLOPs 4.59T vs 8.45T.
• Real-world average success 70.0% on Galaxea R1 Lite vs 51.8% for OpenVLA-OFT and 28.5% for OpenVLA.

Surprising findings

• Removing the training-only dynamic-object and global-depth supervision lowers LIBERO success from 98.1% to 93.3%, indicating the auxiliary reasoning matters at training time only.

Limitations

• Adds about 2B parameters from VGGT, increasing model size relative to the base OpenVLA.
• Real-world evaluation covers only four long-horizon tasks on two platforms with 10-15 trials per task.

Experimental setup

Datasets: LIBERO ManiSkill2 RoboTwin 2.0 real-world teleoperated demonstrations

Benchmarks: LIBERO ManiSkill2 RoboTwin 2.0 Galaxea R1 Lite real-world tasks AgileX Cobot Magic real-world tasks

Metrics: success rate latency throughput FLOPs training cost

Baselines: OpenVLA OpenVLA-OFT SpatialVLA pi0 CoT-VLA Octo CogACT GeoVLA FastV SliME

Builds on: OpenVLA VGGT SigLIP DINOv2 FiLM

Closest papers by meaning

• Localizing, Structuring, and Rendering: Bridging 3D and 2D Vision-Language-Action Models for Robotic Manipulation (CVPR 2026 · similarity 0.91)
• Seeing Across Views: Benchmarking Spatial Reasoning of Vision-Language Models in Robotic Scenes (ICLR 2026 · similarity 0.91)
• 4D-VLA: Spatiotemporal Vision-Language-Action Pretraining with Cross-Scene Calibration (NeurIPS 2025 · similarity 0.90)
• Linear Mechanisms for Spatiotemporal Reasoning in Vision Language Models (ICLR 2026 · similarity 0.90)
• SpatialClaw: Rethinking Action Interface for Agentic Spatial Reasoning (NeurIPS 2026 · similarity 0.90)
• SpinBench: Perspective and Rotation as a Lens on Spatial Reasoning in VLMs (ICLR 2026 · similarity 0.90)