Research map / Video / Video-language understanding
CineTechBench: A Benchmark for Cinematographic Technique Understanding and Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces CineTechBench, an expert-annotated benchmark for cinematographic technique understanding (QA and description of MLLMs across seven dimensions) and camera-movement generation by video generation models.
Problem
Existing movie and video benchmarks focus on high-level semantics, so it is unclear whether MLLMs understand fine-grained cinematographic techniques and whether video generators can reproduce cinema-quality camera movements.
Approach
A taxonomy of seven cinematography dimensions (shot scale, angle, composition, camera movement, lighting, color, focal length) with manually expert-annotated film images and clips, used to evaluate both MLLM understanding (QA and descriptions scored with CAPability-style metrics) and video generators' camera-motion reconstruction (RotErr, TransErr, CamMC, CLIP-IS).
Key results
- GPT-4o achieves the highest overall image QA accuracy among MLLMs at 70.16%, with Gemini-2.5-Pro second at 69.67%.
- Gemini-2.5-Pro achieves the best camera movement QA accuracy at 56.69%, while the best open-source model InternVL3 reaches 55.25% on static image QA.
- Klingv1.6 with first+last frame control gets the best RotError (21.68) and TransError (48.49 relative) among video generators; models show higher error at high rotation angular velocity.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GPT-4o instead of other evaluated models | MLLM static cinematographic QA · Static cinematographic QA | Overall accuracy: 70.16 | Helped |
| Use Gemini-2.5-Pro instead of other evaluated models | MLLM camera movement QA · Camera movement QA | Overall accuracy: 56.69 | Helped |
| Use Gemini-2.5-Pro instead of other evaluated models | MLLM cinematographic description generation · Cinematographic description generation | F1: 52.27 | Helped |
| Use InternVL3 instead of other evaluated open-source models | Open-source MLLM cinematographic description generation · Cinematographic description generation | F1: 47.86 | Helped |
| Use Klingv1.6 with first-frame, last-frame, and text conditions | Commercial video generation models · Camera movement generation | RotError: 21.68 | Helped |
| Use Wan2.1-FLF-14B-720P with first-frame, last-frame, and text conditions | Open-source video generation models · Camera movement generation | TransError Rel.: 48.31 | Helped |
| Generate with high camera rotation angular velocity | Video generation models · Camera movement generation | RotError | Hurt |
| Evaluate camera rotation understanding | MLLMs · Camera movement QA | Accuracy | Hurt |
Surprising findings
- Several open-source MLLMs fail to recognize static shots in camera movement QA (e.g., LLaVA-NeXT-Video), and all models struggle with camera rotation direction.
- MLLMs achieve high hit rates (above 80%) on most description dimensions but F1 scores are only about 30–50%, indicating poor precision.
Limitations
- Annotation of the benchmark is labor-intensive, so scale is limited to about 600 images and 120 clips.
- Camera movement generation evaluation depends on MonST3R trajectory estimation, which adds estimation noise.
- Description evaluation relies on GPT-4.1-nano as judge.
Experimental setup
Datasets: IMDb Top 250 film stills Movieclips YouTube clips
Benchmarks: CineTechBench
Metrics: accuracy BLEU@4 METEOR ROUGE-L CAPability HR CAPability AP CAPability AR CAPability F1 RotErr TransErr
Baselines: GPT-4o Gemini-2.5-Pro Gemini-2.0-Flash Qwen2.5-VL InternVL3 Klingv1.6 Wan2.1-FLF-14B-720P FramePack-FLF2V
Builds on: CameraBench CAPability VBench MonST3R
Closest papers by meaning
- ShotBench: Expert-Level Cinematic Understanding in Vision-Language Models (NeurIPS 2025 · similarity 0.90)
- Stable Cinemetrics : Structured Taxonomy and Evaluation for Professional Video Generation (NeurIPS 2025 · similarity 0.90)
- Unveiling the Value of Motion for Cinematic Camera Trajectories (NeurIPS 2026 · similarity 0.89)
- Auteur: Language-Driven Cinematographic Framing for Human-Centric Video Generation (NeurIPS 2026 · similarity 0.89)
- Towards Understanding Camera Motions in Any Video (NeurIPS 2025 · similarity 0.87)
- CineOrchestra: Unified Entity-Centric Conditioning for Cinematic Video Generation (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.