Research map / Video / Video generation
Identity-Aware Multi-Sentence Video Description
Jae Sung Park, Trevor Darrell, Anna Rohrbach
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Transformer with BERT gender-pretrained text encoding instead of BERT Character LM (ablation) | Fill-in the Identity · LSMDC Validation (Fill-in the Identity) | Class-Acc: 57.9 -> 62.6 | Improved |
| Add training data augmentation with overlapping clip sets (ablation) | Transformer + BERT Gen. LM · LSMDC Validation (Fill-in the Identity) | Class-Acc: 62.6 -> 64.4 | Improved |
| Add training data augmentation with overlapping clip sets (ablation) | Transformer + BERT Gen. LM · LSMDC Validation (Fill-in the Identity) | Gender accuracy: 80.3 -> 81.8 | Improved |
| Add face cluster features (ablation) | Transformer + BERT Gen. LM + Aug. · LSMDC Validation (Fill-in the Identity) | Class-Acc: 64.4 -> 65.3 | Improved |
| Add face cluster features (ablation) | Transformer + BERT Gen. LM + Aug. · LSMDC Validation (Fill-in the Identity) | Inst-Acc: 69.6 -> 69.7 | No clear effect |
| Add I3D video features (ablation) | Transformer + Aug. + Face · LSMDC Validation (Fill-in the Identity) | Class-Acc: 65.3 -> 65.7 | Improved |
| Add I3D video features (ablation) | Transformer + Aug. + Face · LSMDC Validation (Fill-in the Identity) | Inst-Acc: 69.7 -> 69.6 | No clear effect |
| Add visual gender prediction loss (ablation) | Transformer + Aug. + Face + Video · LSMDC Validation (Fill-in the Identity) | Class-Acc: 65.7 -> 65.9 | Improved |
| Add visual gender prediction loss (ablation) | Transformer + Aug. + Face + Video · LSMDC Validation (Fill-in the Identity) | Gender accuracy: 81.8 -> 83.0 | Improved |
| Add visual input to the model (full vs text only) (ablation) | Fill-in the Identity · LSMDC Test (Fill-in the Identity) | Class-Acc: 62.7 -> 64.9 | Improved |
| Let humans see the video | Human annotators · LSMDC Test, 200 random sets (Fill-in the Identity) | Class-Acc: 65.7 -> 86.8 | Improved |
| Use joint Transformer prediction rather than pairwise binary matching (Yu et al., Brown et al.) | Fill-in the Identity · LSMDC Test (Fill-in the Identity) | Class-Acc: 40.6 -> 64.9; 47.5 -> 64.9 | Improved |
Closest papers by meaning
- MICap: A Unified Model for Identity-Aware Movie Descriptions (CVPR 2024 · similarity 0.89)
- ARGUS: Stacked Multi-View Identity Mosaic Injection for Subject-Preserving Video Generation (NeurIPS 2026 · similarity 0.85)
- Identity-Text Video Corpus Grounding (AAAI 2025 · similarity 0.85)
- Illuminating Visual Identity in Universal Multimodal Embeddings (CVPR 2026 · similarity 0.84)
- Movie Weaver: Tuning-Free Multi-Concept Video Personalization with Anchored Prompts (CVPR 2025 · similarity 0.84)
- IDA-VLM: Towards Movie Understanding via ID-Aware Large Vision-Language Model (ICLR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
