Research map / Vision-language and multimodal / Robot action and navigation

PhysVLM-AVR: Active Visual Reasoning for Multimodal Large Language Models in Physical Environments

NeurIPS 2025 · Vision-language and multimodal › Robot action and navigation · approach: Embodied multimodal LLM agents · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces Active Visual Reasoning (AVR), a benchmark (CLEVR-AVR), a CoT dataset (AVR-152k), and PhysVLM-AVR-3B, an MLLM trained to actively gather information in partially observable physical scenes.

Problem

Visual reasoning in MLLMs is studied mostly in passive, fully observable settings, so models fail when occlusion or limited view requires embodied information gathering.

Approach

Formulates active visual reasoning as a closed perception-reasoning-action loop, with a simulated benchmark scoring information sufficiency judgment and information gain, and a dataset whose chain-of-thought annotations explicitly model uncertainty identification, action-conditioned information gain prediction, and action selection.

Key results

• PhysVLM-AVR-3B reaches 90.5% ACC_ISJ, 29.9% IGR and 39.7% ACC_FA on CLEVR-AVR average, versus 88.4/50.8/45.7 for GPT-4o.
• Embodied-Reasoner-7B reaches 20.2% ACC_ISJ but only 1.6% ACC_FA, indicating detection of missing information without effective action.
• Removing AVR-Core drops PhysVLM-AVR-3B ACC_FA from 39.1% to 2.3%; removing CoT drops ACC_FA to 16.9%.

Measured effects reported in the paper

Surprising findings

• Open-source embodied MLLMs can flag information incompleteness (about 20% ACC_ISJ) yet almost never answer correctly (1.6% ACC_FA).
• The 3B PhysVLM-AVR model's final-answer accuracy (39.7%) is close to the 7B AVR-Qwen2.5-VL model's (38.1%) average, and its IGR is lower (29.9% vs 34.7%).

Limitations

• Large gap between ACC_ISJ (90.5%) and ACC_FA (39.7%) shows action selection and multi-step integration remain unsolved.
• Evaluation relies on a simulated benchmark (CLEVR-AVR) with a fixed candidate action set rather than open-ended actions.

Experimental setup

Datasets: CLEVR-AVR AVR-152k LLaVA-Pretrain LLaVA-OneVision-data M4-Instruct-data Reason-RFT-129k AM-DeepSeek-R1-Distilled-100k ScanNet RT1 OpenEQA RoboVQA GeoMath

Benchmarks: CLEVR-AVR OpenEQA RoboVQA GeoMath Geometry3K

Metrics: ACC_ISJ IGR ACC_FA LLM-score BLEU-1 to BLEU-4 Accuracy

Baselines: Qwen2.5-VL-7B LLaVA-OV-7B R1-Onevision-7B Reason-RFT-7B Embodied-Reasoner-7B RoboBrain-7B GPT-4o Gemini-2.0-flash

Builds on: LLaVA CLEVR Genesis

Closest papers by meaning

• VisuRiddles: Fine-grained Perception is a Primary Bottleneck for Multimodal Large Language Models in Abstract Visual Reasoning (ICLR 2026 · similarity 0.91)
• EmbodiedBench: Comprehensive Benchmarking Multi-modal Large Language Models for Vision-Driven Embodied Agents (ICML 2025 · similarity 0.91)
• PhysBench: Benchmarking and Enhancing Vision-Language Models for Physical World Understanding (ICLR 2025 · similarity 0.90)
• QUANTIPHY: A Quantitative Benchmark Evaluating Physical Reasoning Abilities of Vision-Language Models (CVPR 2026 · similarity 0.90)
• V-ABS: Action-Observer Driven Beam Search for Dynamic Visual Reasoning (ICML 2026 · similarity 0.89)
• Seeing Across Views: Benchmarking Spatial Reasoning of Vision-Language Models in Robotic Scenes (ICLR 2026 · similarity 0.89)