Research map / Video / Video-language understanding
Causality Matters: How Temporal Information Emerges in Video Language Models
Yumeng Shi, Quanyu Long, Yin Wu, Wenya Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| remove positional encodings from a single layer after layer 1 (ablation) | Qwen2.5-VL-7B · TempCompass | GT token prob: within +-2% | No clear effect |
| remove positional encodings in the first layer (ablation) | Qwen2.5-VL-7B · TempCompass | GT token prob: up to -60% | Hurt |
| shuffle video token position IDs in layer 1 (ablation) | Qwen2.5-VL-7B · TempCompass MCQ | GT token prob: +0.07% | No clear effect |
| shuffle query token position IDs in layer 1 (ablation) | Qwen2.5-VL-7B · TempCompass MCQ | GT token prob: -53.09% | Hurt |
| reverse temporal position IDs only (ablation) | Qwen2.5-VL-7B · TempCompass | GT token prob: -1.13% to -1.76% | No clear effect |
| reverse frame order keeping original position IDs (ablation) | Qwen2.5-VL-7B · TempCompass MCQ | GT token prob: -15.45% | Hurt |
| block final token attention to query tokens (ablation) | Qwen2.5-VL-7B · TempCompass MCQ | GT token prob: exceeds -40% | Hurt |
| block final token attention to frame tokens (ablation) | Qwen2.5-VL-7B · TempCompass | GT token prob | No clear effect |
| block inter-frame attention (ablation) | Qwen2.5-VL-7B · TempCompass | GT token prob: -4% to -6% near layer 10 | Hurt |
| restrict query attention to Frame 4 only instead of Frame 1 (ablation) | Qwen2.5-VL-7B · TempCompass MCQ | GT token prob: -8% (Frame 1) vs below 2% (Frame 4) | Improved |
| disable frame-to-frame attention in layers 20-28 | Qwen2.5-VL-7B · NExT-QA | accuracy: 75.1 -> 75.2 | No clear effect |
| remove frame tokens from KV cache in layers 20-28 | Qwen2.5-VL-7B · TempCompass Yes/No | accuracy: 70.8 -> 70.7 | No clear effect |
Closest papers by meaning
- Before It Fades: Reinforcing Temporal Representations at Inference Time in VideoLLMs (NeurIPS 2026 · similarity 0.91)
- EventLens: Event-Structure Reinforcement Learning for Video Understanding (NeurIPS 2026 · similarity 0.90)
- Temporal Reasoning Transfer from Text to Video (ICLR 2025 · similarity 0.88)
- Enhancing Temporal Understanding in Video-LLMs through Stacked Temporal Attention in Vision Encoders (NeurIPS 2025 · similarity 0.88)
- SlowFocus: Enhancing Fine-grained Temporal Understanding in Video LLM (NeurIPS 2024 · similarity 0.88)
- Momentor: Advancing Video Large Language Model with Fine-Grained Temporal Reasoning (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
