Research map / Vision-language and multimodal / Multimodal LLMs

Beyond Static Vision: Scene Dynamic Field Unlocks Intuitive Physics Understanding in Multi-modal Large Language Models

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Chain-of-thought multimodal reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces Next Frame Selection and Temporal Coherence Verification benchmarks for fluid intuitive physics in MLLMs, and proposes Scene Dynamic Field (simulator-derived velocity-colored visual prompts) with multi-task SFT that improves fluid dynamics understanding and transfers to other continuum domains.

Problem

Current MLLMs perform near chance on low-level intuitive physics understanding of continuum objects (e.g., fluid dynamics) when evaluated on frame-level dynamics tasks, and high-level physics benchmarks entangle many capabilities.

Approach

Disentangles physical reasoning evaluation into two low-level tasks (NFS and TCV) and enhances MLLMs with Scene Dynamic Fields: simulator-derived velocity-to-blue-channel images used as visual prompts within a multi-task fine-tuning framework (dynamic perception, SDF-guided CoT, original tasks) with expert and self-distilled CoT data.

Key results

• Best open-source zero-shot model Qwen2.5-VL reaches only 30.0% (stride 4) on 4-choice NFS, while GPT-4o reaches 39.79%.
• SDF-Ours improves NFS on the Fluid benchmark by 14.40% on Qwen2-VL and 20.7% on GLM-4.1V over zero-shot.
• Standard fine-tuning performs near zero-shot on transfer domains (e.g., 23.64% vs 22.42% for cloth on Qwen2-VL), while SDF keeps gains.

Surprising findings

• Increasing model scale (e.g., Qwen2-VL 3B to 72B, InternVL2.5 2B to 26B) yields little or inconsistent NFS improvement.
• Optical flow as a visual prompt lowers NFS accuracy relative to no visual prompt (27.0 vs 29.1).

Limitations

• Benchmark and method are restricted mainly to fluid dynamics; generalization to other continuum domains is only exploratory.
• Reported results are on 4 A100 40G GPUs with a relatively small training set (about 3,000 examples in ablation), and some details (main training size, exact SDF rendering) are unclear.
• Relies on simulator fidelity, and real-world videos use LLM-based filtering and human verification that may introduce bias.

Experimental setup

Datasets: ContPhy PhysBench Web-mined real-world fluid videos

Benchmarks: NFS (Next Frame Selection) TCV (Temporal Coherence Verification)

Metrics: accuracy

Baselines: InternVL2.5 InternVL3 GLM-4.1V mPLUG-Owl3 Qwen2-VL Qwen2.5-VL GPT-4o Gemini-2.5-Flash

Builds on: Chain-of-Thought prompting V-JEPA ContPhy PhysBench SWIFT

Closest papers by meaning

• Thinking in Dynamics: How Multimodal Large Language Models Perceive, Track, and Reason Dynamics in Physical 4D World (CVPR 2026 · similarity 0.89)
• PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding (ICLR 2025 · similarity 0.88)
• Multi-SpatialMLLM: Multi-Frame Spatial Understanding with Multi-Modal Large Language Models (CVPR 2026 · similarity 0.88)
• MAPS: Advancing Multi-Modal Reasoning in Expert-Level Physical Science (ICLR 2025 · similarity 0.88)
• SeePhys: Does Seeing Help Thinking? – Benchmarking Vision-Based Physics Reasoning (NeurIPS 2025 · similarity 0.88)
• POINTS-Long: Adaptive Dual-Mode Visual Reasoning in MLLMs (CVPR 2026 · similarity 0.88)