Research map / Video / Video-language understanding
Visual Objectification in Films: Towards a New AI Task for Video Interpretation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces the ObyGaze12 dataset of 1914 expert-annotated film clips and a video-interpretation task for detecting character objectification, benchmarked with X-CLIP/ViViT and concept bottleneck models.
Problem
Detecting interpretive, high-level on-screen objectification of characters (the "male gaze") in films, which arises from complex temporal patterns in cinema and is not captured by presence-based gender analyses.
Approach
Frames objectification as a new interpretive video task, builds a thesaurus of eight visual concepts from film studies and psychology, and densely annotates MovieGraphs clips with four objectification levels including hard negatives, then analyzes feasibility with frozen video-language embeddings and post-hoc concept bottleneck models.
Key results
- X-CLIP reaches F1 0.79 on EN vs. S and 0.82 on (EN∪HN) vs. S when trained on HN vs. S, outperforming ViViT-B/16 (0.53 to 0.73).
- Inter-annotator agreement is gamma=0.42 over all four levels and 0.69 when excluding Not Sure clips.
- Concepts Type of shot, Look, Posture and Appearance are the hardest to separate linearly in X-CLIP embeddings.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train with hard negative examples instead of easy negatives | X-CLIP objectification classifier · ObyGaze12; test (EN ∪ HN) vs S | F1-score: 0.66 -> 0.82 | Helped |
| Train with hard negative examples instead of easy negatives | X-CLIP objectification classifier · ObyGaze12; test EN vs S | F1-score: 0.79 -> 0.71 | Hurt |
| Represent the objectification task using X-CLIP concept bottleneck models instead of the X-CLIP classifier | ObyGaze12 objectification detection · ObyGaze12; test EN vs S | F1-score: 0.79 -> 0.68 | Hurt |
| Use X-CLIP embeddings to represent Type of Shot, Look, Posture, and Appearance concepts | ObyGaze12 concept representation · ObyGaze12 | concept separability | Hurt |
Surprising findings
- Including hard negative examples in training improves classification, and concept detection gets harder when negatives include sure and hard-negative clips without the concept.
Limitations
- Subjective, interpretive annotation that is tedious and hard to scale
- Dataset is small-scale (1914 clips from 12 films)
- Best existing models reach only moderate performance, with concept representations poorly linearly separable
Experimental setup
Datasets: MovieGraphs Kinetics LSMDC
Benchmarks: ObyGaze12
Metrics: F1-score inter-annotator agreement (gamma)
Baselines: ViViT-B/16 X-CLIP PCBM-DT PCBM-LR random all-positive
Builds on: X-CLIP CLIP Post-hoc Concept Bottleneck Models MovieGraphs
Closest papers by meaning
- CineTechBench: A Benchmark for Cinematographic Technique Understanding and Generation (NeurIPS 2025 · similarity 0.84)
- VideoHallu: Evaluating and Mitigating Multi-modal Hallucinations on Synthetic Video Understanding (NeurIPS 2025 · similarity 0.84)
- Evaluating Multimodal Narrative Understanding of Popular Hollywood Films (NeurIPS 2026 · similarity 0.84)
- Causal Tracing of Object Representations in Large Vision Language Models: Mechanistic Interpretability and Hallucination Mitigation (AAAI 2026 · similarity 0.84)
- ViMU: Benchmarking Video Metaphorical Understanding (NeurIPS 2026 · similarity 0.83)
- A Motion-aware Spatio-temporal Graph for Video Salient Object Ranking (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.