Research map / Architectures and efficiency / Attention and transformers
Towards Automated Movie Trailer Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Trailer Generation Transformer (TGT), a seq2seq encoder-decoder that treats movie-to-trailer generation as autoregressive shot-feature regression, and builds MAD and MovieNet trailer benchmarks.
Problem
Automatic trailer generation: selecting and ordering shots from a full movie to form a trailer. Prior classification and ranking formulations suffer from class imbalance, ignore shot composition, and produce chronological rather than reordered shots.
Approach
Formulate trailer generation as a sequence-to-sequence translation of shot-embedding sequences: a movie encoder with a trailerness encoder and context encoder feeds an autoregressive trailer decoder that regresses the next trailer shot embedding, which is then matched to movie shots by nearest-neighbor retrieval.
Key results
- On MAD, TGT reaches 52.38 F1 and 21.18 LD versus CCANet's 31.63 F1 and CLIP-It's 41.73 F1.
- On MovieNet, TGT reaches 46.77 F1 versus CLIP-It's 39.34 and CCANet's 29.58.
- Text conditioning with plot summaries raises MAD F1 from 52.38 to 58.52.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use autoregressive sequence-to-sequence trailer generation instead of shot ranking or independent classification | TGT · MAD | F1-score: 41.73 -> 52.38 | Helped |
| use autoregressive sequence-to-sequence trailer generation instead of shot ranking or independent classification | TGT · MovieNet | F1-score: 39.34 -> 46.77 | Helped |
| add trailerness encoding to the movie sequence (ablation) | TGT · MAD | F1-score: 50.20 -> 52.38 | Helped |
| add trailerness encoding to the movie sequence (ablation) | TGT · MovieNet | F1-score: 44.75 -> 46.77 | Helped |
| add contextualized movie representations with the context encoder (ablation) | TGT · MAD | Recall: 36.07 -> 49.92 | Helped |
| add contextualized movie representations with the context encoder (ablation) | TGT · MovieNet | Recall: 33.41 -> 44.32 | Helped |
| add trailerness loss to reconstruction loss (ablation) | TGT · MAD | Precision: 49.74 -> 52.35 | Helped |
| add trailerness loss to reconstruction loss (ablation) | TGT · MovieNet | Precision: 45.40 -> 46.52 | Helped |
| add KL divergence loss to reconstruction loss (ablation) | TGT · MAD | LD: 26.81 -> 21.99 | Helped |
| add KL divergence loss to reconstruction loss (ablation) | TGT · MovieNet | LD: 31.84 -> 25.19 | Helped |
| add encoded plot summary text (ablation) | TGT · MAD | F1-score: 52.38 -> 56.81 | Helped |
| add contextualized plot summary text (ablation) | TGT · MovieNet | F1-score: 46.77 -> 53.72 | Helped |
Surprising findings
- CCANet achieves a lower Levenshtein distance than TGT in its reported table setting, which the authors attribute to its assumption of equal sequence length to the ground truth.
Limitations
- No dialogue or sound modeling, so fine-editing is not addressed.
- Performance depends on training data size; 10% of data drops F1 substantially.
- Evaluation relies on matching to ground-truth trailer shots, which may penalize plausible alternatives.
Experimental setup
Datasets: MAD MovieNet IMDb trailers
Benchmarks: MAD MovieNet
Metrics: Precision Recall F1-score Levenshtein distance sequence length difference
Baselines: Random CCANet CLIP-It
Builds on: Transformer CLIP TransNet-V2 CLIP-It CCANet
Closest papers by meaning
- MovieDreamer: Hierarchical Generation for Coherent Long Visual Sequences (ICLR 2025 · similarity 0.82)
- TeaserGen: Generating Teasers for Long Documentaries (ICLR 2025 · similarity 0.82)
- SEINE: Short-to-Long Video Diffusion Model for Generative Transition and Prediction (ICLR 2024 · similarity 0.82)
- MovieBench: A Hierarchical Movie Level Dataset for Long Video Generation (CVPR 2025 · similarity 0.82)
- StreamingT2V: Consistent, Dynamic, and Extendable Long Video Generation from Text (CVPR 2025 · similarity 0.82)
- Mimir: Improving Video Diffusion Models for Precise Text Understanding (CVPR 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.