Research map / Video / Video-language understanding
TSPO: Temporal Sampling Policy Optimization for Long-form Video Language Understanding
Canhui Tang, Zifan Han, Hongbo Sun, Sanping Zhou, Xuchong Zhang, Xin Wei, Ye Yuan, Huayu Zhang, Jinglin Xu, Hao Sun
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add TSPO temporal agent to LLaVA-Video-7B (64 frames) | LLaVA-Video-7B · MLVU Dev | accuracy: 70.3 -> 76.3 | Improved |
| add TSPO temporal agent to LLaVA-Video-7B (64 frames) | LLaVA-Video-7B · Video-MME (Average) | accuracy: 64.4 -> 65.5 | Improved |
| add TSPO to LLaVA-Video-7B | LLaVA-Video-7B · LVBench | accuracy: 40.2 -> 45.3 | Improved |
| transfer TSPO agent trained on LLaVA-Video to Qwen2.5-VL without retraining | Qwen2.5-VL-7B · MLVU Dev | accuracy: 65.1 -> 74.3 | Improved |
| transfer TSPO agent to LLaVA-Video-72B | LLaVA-Video-72B · MLVU Dev | accuracy: 74.4 -> 77.3 | Improved |
| train on Comprehensive Temporal data (ablation) | TSPO (LLaVA-Video-7B) · LongVideoBench / Video-MME | accuracy: 58.9 -> 62.8 / 64.4 -> 65.5 | Improved |
| train on Needle-in-a-Haystack data only (ablation) | TSPO (LLaVA-Video-7B) · LongVideoBench / Video-MME | accuracy: 58.9 -> 63.4 / 64.4 -> 64.6 | Mixed |
| combine both data styles (ablation) | TSPO (LLaVA-Video-7B) · LongVideoBench / Video-MME | accuracy: 63.8 / 65.0 (accuracy reward only) | Improved |
| add temporal locating reward to accuracy reward (ablation) | TSPO (LLaVA-Video-7B) · LongVideoBench / Video-MME | accuracy: 63.8 / 65.0 -> 63.9 / 65.5 | Improved |
| use RL (TSPO) instead of end-to-end SFT with Gumbel-Softmax (ablation) | temporal agent · LongVideoBench / Video-MME | accuracy: 62.8 / 64.8 -> 63.9 / 65.5 | Improved |
| use TSPO selector instead of CoS 13B selector | LLaVA-Video-7B · Frame selection time (128 candidates) | frame selection time: 28.4 -> 1.2 | Improved |
| reduce frames from 64 to 32 with TSPO (ablation) | LLaVA-Video + TSPO · LongVideoBench / Video-MME | accuracy: 60.6 / 65.3 -> 59.6 / 64.8 | Mixed |
Closest papers by meaning
- Adaptive Keyframe Sampling for Long Video Understanding (CVPR 2025 · similarity 0.89)
- Incentivizing Versatile Video Reasoning in MLLMs via Data-Efficient Reinforcement Learning (CVPR 2026 · similarity 0.89)
- Task Preference Optimization: Improving Multimodal Large Language Models with Vision Task Alignment (CVPR 2025 · similarity 0.89)
- DeepVideo-R1: Video Reinforcement Fine-Tuning via Difficulty-aware Regressive GRPO (NeurIPS 2025 · similarity 0.88)
- Self-alignment of Large Video Language Models with Refined Regularized Preference Optimization (NeurIPS 2025 · similarity 0.88)
- EventLens: Event-Structure Reinforcement Learning for Video Understanding (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
