Research map / Vision-language and multimodal / Multimodal LLMs

EgoProx: Evaluating MLLMs on Egocentric 3D Proximity Reasoning Across a Cognitive Hierarchy

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Spatial reasoning in VLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces EgoProx, a multiple-choice/chain-of-actions benchmark for egocentric 3D proximity reasoning built with an agentic data engine, evaluates MLLMs, and shows LoRA tuning on small cross-task/cross-dataset data improves performance.

Problem

Whether multimodal LLMs can reason about 3D proximity between the camera wearer's body and surrounding objects from egocentric video, across intention, exploration, exploitation, and chain-of-actions tasks.

Approach

A cognitive-hierarchy benchmark for egocentric 3D proximity, with QA pairs generated by an agent (Gemini-2.5-Pro) orchestrating a 3D analysis toolset (occupancy maps, A* paths, spatial calculators, gaze parser, affordance detector), plus a probe showing small task-specific LoRA tuning yields cross-task gains.

Key results

• Gemini-2.5-Pro reaches 50.24 on Exploitation approximate and 25.14 Act-Acc on Chain of Actions, far below the human level of 80.23 on Act-Acc.
• Qwen2.5-VL-7B with intention-only LoRA tuning raises Exploitation approximate accuracy from 38.63 to 64.93.
• Qwen2.5-VL-7B fine-tuned on ADT raises EgoExo4D Intention accuracy from 26.74 to 50.58 (relative to zero-shot), showing large cross-dataset gains.

Surprising findings

• Intention-category tuning transfers more strongly to other tasks than tuning on exploration or exploitation data.
• Scaling Qwen2.5-VL from 7B to 72B yields only limited gains on EgoProx.

Limitations

• Benchmark is primarily indoor daily activities, with limited outdoor and rare scenarios.
• Agent pipeline relies on video metadata such as camera poses and 3D bounding boxes, limiting applicability to other datasets.
• Does not evaluate specialized spatial reasoning models in main results.

Experimental setup

Datasets: EgoExo4D Aria Digital Twin (ADT)

Benchmarks: EgoProx

Metrics: accuracy Act-Acc Rel-Acc-S Rel-Acc-L

Baselines: GPT-5 Gemini-2.5-Pro LLaVA-NeXT-Video-7B MiniCPM-V 2.6 InternVL 2.5-8B Qwen2.5-VL-7B Qwen2.5-VL-32B Qwen2.5-VL-72B Qwen3-VL-235B Qwen3-VL-Plus

Builds on: EgoExo4D Aria Digital Twin VSI-Bench LLaMA-Factory

Closest papers by meaning

• Spatial Reasoning with Vision-Language Models in Ego-Centric Multi-View Scenes (ICLR 2026 · similarity 0.91)
• EgoMind: Activating Spatial Cognition through Linguistic Reasoning in MLLMs (CVPR 2026 · similarity 0.91)
• EgoThinker: Unveiling Egocentric Reasoning with Spatio-Temporal CoT (NeurIPS 2025 · similarity 0.90)
• From Where Things Are to What They Are For: Benchmarking Spatial-Functional Intelligence in Multimodal LLMs (CVPR 2026 · similarity 0.90)
• Seeing Together: Multi-Robot Cooperative Egocentric Spatial Reasoning with Multimodal Large Language Models (NeurIPS 2026 · similarity 0.90)
• Abstract 3D Perception for Spatial Intelligence in Vision-Language Models (CVPR 2026 · similarity 0.89)