Research map / Video / Video-language understanding
ExAct: A Video-Language Benchmark for Expert Action Analysis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces ExAct, a 3,521-question multiple-choice video-language benchmark for expert-level understanding of skilled physical activities, built from Ego-Exo4D expert commentaries with LLM-generated distractors and expert review.
Problem
Existing video-language and action recognition benchmarks do not test expert-level, fine-grained understanding of skilled physical human activities, and skill-assessment datasets lack open-ended, evaluable language annotations.
Approach
Rewrites noisy Ego-Exo4D expert commentaries into concise feedback statements, generates four plausible distractors with LLMs using action-replacement and reasoning-error strategies, filters language-only shortcuts with length constraints and blind-LLM tests, and has domain experts verify each item, yielding a 5-way MCQ benchmark.
Key results
- Gemini 2.5 Pro is the best-performing model at 55.35% accuracy on ExAct.
- Non-expert humans achieve 61.86% and domain experts achieve 82.02% accuracy.
- The final benchmark contains 3,521 QA pairs over 11 activities in 6 domains, filtered from over 400k Ego-Exo4D commentaries.
Surprising findings
- Non-expert humans (61.86%) outperform the best VLM (Gemini 2.5 Pro, 55.35%) on the benchmark.
- Only about 0.9% of the original commentaries survive the full filtering and expert verification pipeline.
Limitations
- Results come from single deterministic inference runs with no error bars.
- Videos include identifiable human faces, raising privacy concerns.
Experimental setup
Datasets: Ego-Exo4D expert commentaries
Benchmarks: ExAct
Metrics: multiple-choice accuracy
Baselines: Gemini 1.5 Pro GPT-4o GPT-4.1 LLaVA-Video LLaVA-OneVision Qwen2.5-VL VideoLLaMA InternVL2.5 PerceptionLM
Builds on: Ego-Exo4D
Closest papers by meaning
- ExpertAF: Expert Actionable Feedback from Video (CVPR 2025 · similarity 0.88)
- ActionAtlas: A VideoQA Benchmark for Domain-specialized Action Recognition (NeurIPS 2024 · similarity 0.86)
- Learning Skill-Attributes for Transferable Assessment in Video (NeurIPS 2025 · similarity 0.86)
- SLVMBench: Skill Learning from Video Memory (NeurIPS 2026 · similarity 0.85)
- SportR: A Benchmark for Multimodal Large Language Model Reasoning in Sports (ICLR 2026 · similarity 0.85)
- SPORTU: A Comprehensive Sports Understanding Benchmark for Multimodal Large Language Models (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.