AI Paper Atlas

Research map / Video / Video-language understanding

EgoSchema: A Diagnostic Benchmark for Very Long-form Video Language Understanding

Karttikeya Mangalam, Raiymbek Akshulakov, Jitendra Malik

NeurIPS 2023 · Video › Video-language understanding · approach: Video question answering · 45 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Test humans in the Video → Text setting instead of No constraintHuman QA on EgoSchema · EgoSchemaQA accuracy: 75.0% -> 76.2%Improved
Allow three minutes instead of under one minuteHuman QA on EgoSchema · EgoSchemaQA accuracy: +1.0%Improved
Use 1 fps video (180 frames) for human answeringHuman QA on EgoSchema · EgoSchemaQA accuracy: 67.2%Mixed
Increase the number of input framesInternVideo · EgoSchemaQA accuracyImproved
Increase the number of input framesmPLUG-Owl · EgoSchemaQA accuracyMixed
Generate questions, correct answers, and wrong answers jointly in two chained calls rather than generating answers in separate stagesQ(AW)-shot generation · EgoSchema question-answer generationGenerated answer qualityImproved
Use Q(AW)-shot instead of QAW-shotQuestion-answer generation pipeline · EgoSchema question-answer generationGeneration cost: ~30% cheaperImproved
Generate all question-answer triplets in one inference call instead of using staged promptingOne-shot generation · EgoSchema question-answer generationGeneration qualityHurt
Use an LLM to guess the answer from narrations without the questionNo-Q baseline · EgoSchema question-answer generationAnswer-guessing accuracy: ~20%No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy