Research map / Video / Video-language understanding
EgoExoBench: A Benchmark for First- and Third-person View Video Understanding in MLLMs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces EgoExoBench, a 7,330-question 4-way MCQ benchmark for cross-view (egocentric-exocentric) video understanding in MLLMs, finding strong single-view models struggle on cross-view tasks.
Problem
Whether multimodal LLMs can reason across first-person (egocentric) and third-person (exocentric) video views, which existing video benchmarks do not evaluate.
Approach
A benchmark built from six paired ego-exo datasets with 11 subtasks grouped into semantic alignment (relation), viewpoint association (view transition), and temporal reasoning, using annotation-derived, LLM-generated, and human-annotated MCQs with text-only filtering, and evaluated zero-shot on 13 MLLMs.
Key results
- Best open-source model Qwen2.5-VL-72B reaches 44.7% average (48.5% in one table version) and best closed model GPT-o4-mini 48.0% overall, versus 90.1% human accuracy with 3 minutes per question.
- Chain-of-thought prompting degrades performance on most subtasks; Qwen2.5-VL-72B drops 20% on Person Relation with CoT.
- Adding the reference exocentric video boosts Action Prediction for Qwen2.5-VL-72B by 8.2 points but lowers Skill Evaluation by 1.5 points.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add a cross-perspective reference video (ablation) | Qwen2.5-VL-72B · Action Prediction | accuracy: 31.7 -> 39.9 (+8.2) | Helped |
| Add a cross-perspective reference video (ablation) | InternVL3-78B · Action Prediction | accuracy: 41.9 -> 51.0 (+9.1) | Helped |
| Add a cross-perspective reference video (ablation) | GPT-4o · Action Prediction | accuracy: 26.1 -> 27.7 (+1.1) | Helped |
| Add a cross-perspective reference video (ablation) | Qwen2.5-VL-72B · Skill Evaluation | accuracy: 31.4 -> 29.9 (-1.5) | Hurt |
| Add a cross-perspective reference video (ablation) | InternVL3-78B · Skill Evaluation | accuracy: 30.4 -> 29.5 (-0.9) | Hurt |
| Add a cross-perspective reference video (ablation) | GPT-4o · Skill Evaluation | accuracy: 26.5 -> 26.3 (-0.2) | Hurt |
| Append “Let’s think step by step” to the prompt (ablation) | Qwen2.5-VL-72B · Person Relation | accuracy: -20% | Hurt |
| Append “Let’s think step by step” to the prompt (ablation) | Qwen2.5-VL-72B · Action Relation | accuracy: -19% | Hurt |
| Append a chain-of-thought prompt (ablation) | LLaVA-OV-7B, Qwen2.5-VL-32B, Qwen2.5-VL-72B, and GPT-4o · EgoExoBench subtasks | accuracy | Hurt |
| Increase model size from 7B to 72B | Qwen2.5-VL · EgoExoBench | accuracy: 32.8 -> 44.7 | Helped |
| Use Gemini 2.5 Pro instead of GPT-o4-mini | EgoExoBench · EgoExoBench | accuracy: 48.0 -> 51.7 | Helped |
Surprising findings
- Chain-of-thought prompting hurts performance on most cross-view subtasks, suggesting text-only reasoning disrupts visual-linguistic interleaving.
- EgoGPT, fine-tuned on egocentric data, offers only marginal gains over similarly sized models.
Limitations
- The benchmark may not fully reflect the breadth of real-world ego-exo scenarios.
- Evaluation is zero-shot MCQ only, without fine-tuning or open-ended answers.
Experimental setup
Datasets: Ego-Exo4D EgoExoLearn LEMMA EgoMe TF2023 CVMHAT
Benchmarks: EgoExoBench Video-MME MLVU LongVideoBench CG-Bench EgoSchema
Metrics: accuracy
Baselines: GPT-4o GPT-o4-mini Claude-3.7-Sonnet Gemini 2.5 Pro InternVL3 LLaVA-OV LLaVA-Video NVILA EgoGPT
Builds on: Ego-Exo4D EgoExoLearn LEMMA EgoMe Video-MME
Closest papers by meaning
- EgoThink: Evaluating First-Person Perspective Thinking Capability of Vision-Language Models (CVPR 2024 · similarity 0.90)
- Omnia de EgoTempo: Benchmarking Temporal Understanding of Multi-Modal LLMs in Egocentric Videos (CVPR 2025 · similarity 0.90)
- EgoThinker: Unveiling Egocentric Reasoning with Spatio-Temporal CoT (NeurIPS 2025 · similarity 0.90)
- HumanPCR: Probing MLLM Capabilities in Diverse Human-Centric Scenes (ICLR 2026 · similarity 0.89)
- Retrieval-Augmented Egocentric Video Captioning (CVPR 2024 · similarity 0.89)
- Viewpoint Rosetta Stone: Unlocking Unpaired Ego-Exo Videos for View-invariant Representation Learning (CVPR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.