Research map / Video / Video-language understanding
VITED: Video Temporal Evidence Distillation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Synthesizes temporally grounded evidence chains for existing VideoQA data via hierarchical evidence pools and LLM beam search, then distills them into TimeChat and LLaVA-Video via two-stage training (VITED).
Problem
Video QA models uniformly sample frames and cannot localize and chain multiple pieces of temporally distributed evidence needed for complex questions; existing grounding models require the evidence text in advance.
Approach
Automatically construct chain-of-evidence data by generating hierarchical multi-granularity segment captions, searching for evidence sequences that maximize the LLM's likelihood of the correct answer, summarizing them into timestamped CoT, and training a video VLM to generate evidence then answer.
Key results
- VITED built on LLaVA-Video (7B) reaches 25.19 points higher than LLaVA-Video on NExT-GQA Acc@GQA and 2.62 points higher on CinePile, achieving SOTA on 4 of 6 VideoQA benchmarks among same-size models.
- On NExT-GQA, VITED (TimeChat-based) achieves IoP@0.5 of 41.22 and Acc@GQA of 27.61%, outperforming the GPT-4 driven LLoVi agent (38.0 / 26.8).
- Human evaluation scores of synthetic evidence chains exceed 80.4% average across five aspects; a text-only LLM answers NExT-QA at 72.05% from evidence chains versus 51.71% from conventional CoT.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Distill temporal evidence chains instead of using LLaVA-Video without evidence distillation | LLaVA-Video · CinePile | MCQ accuracy: 53.77 -> 56.39 | Helped |
| Distill temporal evidence chains instead of using LLaVA-Video without evidence distillation | LLaVA-Video · STAR | MCQ accuracy: 66.88 -> 68.23 | Helped |
| Distill temporal evidence chains instead of using LLaVA-Video without evidence distillation | LLaVA-Video · MVBench | MCQ accuracy: 58.60 -> 60.95 | Helped |
| Add temporal evidence distillation rather than standard video instruction tuning | TimeChat · PerceptionTest | MCQ accuracy: 57.39 -> 63.66 | Helped |
| Add temporal evidence distillation rather than standard video instruction tuning | TimeChat · NExT-QA | MCQ accuracy: 71.05 -> 73.42 | Helped |
| Add temporal evidence distillation rather than standard video instruction tuning | TimeChat · STAR | MCQ accuracy: 68.39 -> 70.99 | Helped |
| Add temporal evidence distillation rather than standard video instruction tuning | TimeChat · NExT-GQA | Acc@GQA: 0.00 -> 27.61 | Helped |
| Add the evidence pool (ablation) | VITED evidence-generation framework · NExT-QA | Avg.: 71.05 -> 76.14 | Helped |
| Add chain-of-thought summarization (ablation) | VITED evidence-generation framework · NExT-QA | Avg.: 73.80 -> 76.14 | Helped |
| Add Stage-1 video instruction tuning before Stage-2 evidence fine-tuning (ablation) | TimeChat VITED · NExT-QA | Avg.: 71.76 -> 70.99 | Hurt |
| Add hierarchical evidence-pool granularity (ablation) | VITED evidence-generation framework · NExT-QA | Avg.: 74.28 -> 76.14 | Helped |
| Add evidence-chain search (ablation) | VITED evidence-generation framework · NExT-QA | Avg.: 73.22 -> 76.14 | Helped |
Surprising findings
- Naive chain-of-thought prompting gives little or no gain over base video VLMs, and ground-truth-guided direct evidence generation performs worse than unguided variants.
- Video instruction tuning alone gives zero NExT-GQA grounding accuracy, so evidence grounding emerges only from the evidence distillation stage.
Limitations
- Evidence quality depends on hallucinations and vagueness of the base VLM used for the evidence pool.
- Evidence quality degrades on datasets with longer videos (e.g., CinePile) per human evaluation.
Experimental setup
Datasets: NExT-QA STAR PerceptionTest CinePile MVBench NExT-GQA
Benchmarks: CinePile PerceptionTest NExT-QA STAR MVBench NExT-GQA
Metrics: MCQ accuracy IoP@0.5 Acc@GQA
Baselines: TimeChat LLaVA-Video SeViLA LLaMA-3.2V InternVideo2 LLaVA-OneVision LongVA VideoLLaVA LLoVi FrozenBiLM
Builds on: TimeChat LLaVA-Video chain-of-thought prompting LoRA
Closest papers by meaning
- VideoEspresso: A Large-Scale Chain-of-Thought Dataset for Fine-Grained Video Reasoning via Core Frame Selection (CVPR 2025 · similarity 0.89)
- Temporal Chain of Thought: Long-Video Understanding by Thinking in Frames (NeurIPS 2025 · similarity 0.88)
- MoReVQA: Exploring Modular Reasoning Models for Video Question Answering (CVPR 2024 · similarity 0.88)
- Commonsense Video Question Answering through Video-Grounded Entailment Tree Reasoning (CVPR 2025 · similarity 0.88)
- Enhancing Video-LLM Reasoning via Agent-of-Thoughts Distillation (CVPR 2025 · similarity 0.87)
- LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.