Research map / Video / Video-language understanding
Towards Understanding Camera Motions in Any Video
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces CameraBench, a ~3,000-video expert-annotated camera motion dataset and benchmark with a motion-primitive taxonomy and training program, and shows that SFT of Qwen2.5-VL on it substantially improves camera motion understanding.
Problem
Camera motion understanding in internet videos is poorly specified and annotated in existing datasets, and both SfM/SLAM and video-language models struggle to capture camera motion, whether geometric or semantic.
Approach
A cinematographer-designed taxonomy of 50+ camera motion primitives, a label-then-caption annotation framework that permits "not sure" answers, and a tutorial-based annotator training program with expert oversight; the resulting data is used to benchmark SfM/SLAM and VLMs and to SFT Qwen2.5-VL.
Key results
- Qwen2.5-VL-7B SFT reaches 60.0 average AP on motion primitive classification, roughly matching MegaSAM (50.1) and far above the best generative VLM baseline GPT-4o (36.4).
- On VQA, the Qwen2.5-VL-72B SFT model reaches 74.5/49.9 Acc/Q-Acc on motion-steadiness-style skills overall average 62.8 AP-style primitive classification, while most open VLMs score at or below chance.
- Among SfM/SLAM methods, MegaSAM achieves 50.1 average AP, outperforming COLMAP (27.3) but all methods remain around 50% AP.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Supervised fine-tuning on CameraBench videos | Qwen2.5-VL-72B · CameraBench camera-centric motion primitives | Average Precision: 37.7 -> 62.8 | Helped |
| Supervised fine-tuning on CameraBench videos | Qwen2.5-VL-72B · CameraBench VQA | Overall accuracy: 60.3 -> 73.6 | Helped |
| Supervised fine-tuning on CameraBench videos | Qwen2.5-VL-72B · CameraBench VQA | Overall question accuracy: 27.4 -> 47.1 | Helped |
| Supervised fine-tuning on CameraBench videos | Qwen2.5-VL-72B · CameraBench video-text retrieval, skill-based task | Group score: 32.1 -> 44.4 | Helped |
| Supervised fine-tuning on CameraBench videos | Qwen2.5-VL-72B · CameraBench video-text retrieval, caption-based task | Group score: 56.4 -> 78.1 | Helped |
Surprising findings
- Most open-source VQA models perform at or below chance on CameraBench.
- Prior benchmarks such as VDC, DREAM-1K and VidComposition contain high error rates in camera-motion descriptions (e.g., over 55% inaccurate on a VidComposition audit).
Limitations
- No methods fully solve camera motion classification; best AP is around 50%.
- Roll and other rotation primitives remain challenging for VLMs due to long-tailed distribution.
- Camera motion control in video generation is left as future work; the paper only offers a preliminary study.
Experimental setup
Datasets: CameraBench
Benchmarks: CameraBench
Metrics: average precision accuracy question accuracy (Q-Acc) text/image/group score SPICE LLM-as-a-judge
Baselines: COLMAP VGGSfM DUSt3R MASt3R CUT3R MegaSAM InternVideo2 LLaVA-Video LLaVA-OneVision GPT-4o Gemini-2.5-Pro InternVL3-78B
Builds on: VQAScore COLMAP MegaSAM Qwen2.5-VL
Closest papers by meaning
- MotionBench: Benchmarking and Improving Fine-grained Video Motion Understanding for Vision Language Models (CVPR 2025 · similarity 0.90)
- MMBench-Video: A Long-Form Multi-Shot Benchmark for Holistic Video Understanding (NeurIPS 2024 · similarity 0.89)
- ShotBench: Expert-Level Cinematic Understanding in Vision-Language Models (NeurIPS 2025 · similarity 0.89)
- FAVOR-Bench: A Comprehensive Benchmark for Fine-Grained Video Motion Understanding (NeurIPS 2025 · similarity 0.88)
- MVBench: A Comprehensive Multi-modal Video Understanding Benchmark (CVPR 2024 · similarity 0.88)
- CineTechBench: A Benchmark for Cinematographic Technique Understanding and Generation (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.