Research map / Video / Video-language understanding
LongVPO: From Anchored Cues to Self-Reasoning for Long-Form Video Preference Optimization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LongVPO, a two-stage DPO framework that extends short-context VLMs (InternVL2.5-8B, InternVideo2.5) to long videos using ~16K synthetic preference pairs with no long-video labels.
Problem
Short-context vision-language models degrade on ultra-long video understanding, and long-video annotations are costly, while DPO-based long-video alignment assumes a reference model that already handles long contexts.
Approach
Stage 1 builds anchor-based synthetic preference triples by concatenating short clips with distractors, filters by visual similarity and question specificity, and approximates the reference model's long-context likelihood by scoring only the anchor clip. Stage 2 applies self-training DPO on recursively captioned long videos, with LLM-generated multi-scene queries and degraded-context dispreferred
Key results
- LongVPO on InternVL2.5-8B raises LVBench from 45.2 to 50.1 and MLVU from 67.6 to 74.1 using 16K synthetic samples.
- LongVPO on InternVideo2.5 (256f) reaches 51.0 on LVBench and 67.2 on LongVideoBench, surpassing its base model.
- On MVBench, LongVPO Stage 2 (InternVL2.5-based) reaches 73.1, a +1.1 gain over the base model.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add LongVPO Stage1 and Stage2 | InternVL2.5-8B · LVBench, LongVideoBench, MLVU, Video-MME, and MVBench | accuracy: 45.2 -> 50.1; 62.7 -> 66.6; 67.6 -> 74.1; 61.1 -> 64.6; 65.3 -> 70.3; 72.0 -> 73.1 | Helped |
| Add LongVPO Stage1 and two Stage2 iterations | InternVideo2.5-8B · LVBench, LongVideoBench, MLVU, and Video-MME | accuracy: 47.4 -> 51.0; 63.2 -> 67.2; 72.8 -> 74.7; 63.3 -> 66.1 | Helped |
| Add Stage 1 scene filtering (ablation) | LongVPO Stage 1 · MLVU, LongVideoBench, and LVBench | accuracy: 69.8 -> 72.9; 64.2 -> 66.1; 43.4 -> 45.3 | Helped |
| Replace Stage 1 scene filtering with TopK selection (ablation) | LongVPO Stage 1 · MLVU, LongVideoBench, and LVBench | accuracy: 72.9 -> 69.9; 66.1 -> 58.4 | Hurt |
| Use Qwen2.5-32B-generated chosen responses instead of self-generated responses (ablation) | LongVPO Stage 2 · MLVU, LongVideoBench, and LVBench | accuracy: 72.9 -> 73.1; 66.1 -> 65.6; 45.3 -> 44.4 | Mixed |
| Use self-generated responses with scene-interleaved captions (ablation) | LongVPO Stage 2 · MLVU, LongVideoBench, and LVBench | accuracy: 72.9 -> 73.0; 66.1 -> 66.1; 45.3 -> 44.7 | Mixed |
| Replace Qwen2.5-32B with InternLM2.5-7B as the knowledge-transfer LLM (ablation) | LongVPO Stage 2 · MLVU, LongVideoBench, and LVBench | accuracy: 73.1 -> 72.5; 65.6 -> 65.8; 44.4 -> 44.9 | Hurt |
| Use single-video caption data for fine-tuning (ablation) | InternVL2.5-8B · Long-video benchmarks | performance | Hurt |
| Apply SFT to synthesized Stage 1 data instead of single-video caption data (ablation) | InternVL2.5-8B · Long-video benchmarks | performance | Helped |
| Apply DPO to synthesized Stage 1 preference pairs instead of SFT (ablation) | InternVL2.5-8B · Long-video benchmarks | performance | Helped |
| Use complete long videos as input to the reference model instead of the proposed reference approximation (ablation) | LongVPO Stage 1 · General and long-video benchmarks | performance | Hurt |
| Increase Stage 1 synthetic-video input frames (ablation) | LongVPO · Long-video benchmarks | performance | Helped |
Surprising findings
- Using Qwen2.5-32B-generated or scene-interleaved responses as chosen responses in Stage 2 performed worse than self-generated responses.
- Replacing the 7B InternLM2.5 for query generation instead of Qwen2.5-32B causes only a slight drop, suggesting the method does not depend on a large LLM.
Limitations
- Prioritizes performance over inference computational efficiency; no integration with context compression yet.
- No error bars or statistical significance analysis due to limited compute.
- Relies on a proprietary-scale external LLM (Qwen2.5-32B) for query generation in Stage 2.
Experimental setup
Datasets: LLaVA-Video-178K Vript MVBench LVBench LongVideoBench MLVU Video-MME
Benchmarks: LVBench LongVideoBench MLVU Video-MME MVBench
Metrics: accuracy
Baselines: InternVL2.5 InternVideo2.5 VideoChat-Flash LLaVA-Video Qwen2-VL NVILA Kangaroo LongVU
Builds on: DPO InternVL2.5 InternVideo2.5 DeepSpeed Ulysses DINOv2
Closest papers by meaning
- Self-alignment of Large Video Language Models with Refined Regularized Preference Optimization (NeurIPS 2025 · similarity 0.91)
- LongVideoBench: A Benchmark for Long-context Interleaved Video-Language Understanding (NeurIPS 2024 · similarity 0.90)
- SEAL: Semantic Attention Learning for Long Video Representation (CVPR 2025 · similarity 0.89)
- VidChain: Chain-of-Tasks with Metric-based Direct Preference Optimization for Dense Video Captioning (AAAI 2025 · similarity 0.89)
- $\mathcal{V}ista\mathcal{DPO}$: Video Hierarchical Spatial-Temporal Direct Preference Optimization for Large Video Models (ICML 2025 · similarity 0.88)
- Temporal Chain of Thought: Long-Video Understanding by Thinking in Frames (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.