Research map / Video / Video-language understanding

DELTAVID: Enhancing Fine-Grained Spatiotemporal Perception with Cross-Video Differences

NeurIPS 2026 · Video › Video-language understanding · approach: Long video understanding · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DeltaVid, training Video MLLMs with GRPO on synthetic cross-video spot-the-difference pairs (Grounding + MCQ rewards), improving Qwen3-VL-8B on its own diagnostic benchmark and several general video benchmarks.

Problem

Video MLLMs lack precise local spatiotemporal perception: when two near-identical videos differ only in a short time span or small region, they fail to find the change and give reliable evidence.

Approach

Turn controllable programmatic perturbations on real videos into A/B pairs with exact temporal, spatial, and attribute labels, then post-train with rule-based rewards (temporal/spatial IoU for Grounding, exact match for MCQ) via GRPO, so the model learns local evidence from cross-video differences.

Key results

• On DeltaVid-Bench, DeltaVid-Qwen3-VL-8B reaches 67.40 overall vs 58.60 for base Qwen3-VL-8B and 65.20 for Qwen3-VL-235B, with T-IoU rising from 40.57 to 64.00.
• General average across 8 reported benchmarks rises from 60.56 (Qwen3-VL-8B) to 63.06, with TempCompass gaining from 69.38 to 76.76.
• GPT-5 scores highest overall on DeltaVid-Bench (70.00), above DeltaVid's 67.40.

Surprising findings

• Grounding-only and MCQ-only training both lag the full model, and the 8B model trained on synthetic data surpasses the 235B model on the in-domain benchmark.
• S-IoU is essentially unchanged (69.78 base vs 69.33) while gains concentrate on temporal localization.

Limitations

• Differences are programmatic (synthetic overlays and edits), not natural or semantic changes.
• DeltaVid-Bench is small (500 questions) and has per-category counts that are small, so differences may be noisy.
• Some general benchmark baseline numbers (e.g., Video-R1) are taken from original papers, and several entries are missing.

Experimental setup

Datasets: DeltaVid-10K

Benchmarks: DeltaVid-Bench MMVU MLVU MVBench Video-MME VideoHolmes VideoMMMU LVBench TempCompass LongVideoBench

Metrics: MCQ accuracy T-IoU S-IoU overall score accuracy

Baselines: Qwen3-VL-8B Qwen3-VL-235B GPT-5 Video-R1

Builds on: GRPO (DeepSeekMath) Video-R1 Perception-R1 Jigsaw-R1 ViCrit

Closest papers by meaning

• MMBench-Video: A Long-Form Multi-Shot Benchmark for Holistic Video Understanding (NeurIPS 2024 · similarity 0.90)
• VideoRefer Suite: Advancing Spatial-Temporal Object Understanding with Video LLM (CVPR 2025 · similarity 0.89)
• SF2T: Self-supervised Fragment Finetuning of Video-LLMs for Fine-Grained Understanding (CVPR 2025 · similarity 0.89)
• SciVideoBench: Benchmarking Scientific Video Reasoning in Large Multimodal Models (ICML 2026 · similarity 0.88)
• MVBench: A Comprehensive Multi-modal Video Understanding Benchmark (CVPR 2024 · similarity 0.88)
• SlowFocus: Enhancing Fine-grained Temporal Understanding in Video LLM (NeurIPS 2024 · similarity 0.88)