Research map / Video / Video-language understanding
Adversarial Inference for Multi-Sentence Video Description
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use hybrid-discriminator inference instead of MLE decoding | MLE video-description generator · ActivityNet Captions validation | human evaluation delta: -4.0 -> +6.5 | Helped |
| use a hybrid rather than single discriminator during inference | MLE video-description generator · ActivityNet Captions validation | CIDEr-D: 18.17 -> 20.60 | Helped |
| add a pairwise discriminator to the single discriminator | MLE video-description generator with SingleDis · ActivityNet Captions validation | CIDEr-D: 18.17 -> 18.72 | Helped |
| add the visual discriminator (ablation) | MLE + HybridDis w/o Vis · ActivityNet Captions validation | CIDEr-D: 17.29 -> 20.60 | Helped |
| add the language discriminator (ablation) | MLE + HybridDis w/o Lang · ActivityNet Captions validation | CIDEr-D: 19.44 -> 20.60 | Helped |
| add the pairwise discriminator (ablation) | MLE + HybridDis w/o Pair · ActivityNet Captions validation | CIDEr-D: 19.39 -> 20.60 | Helped |
| remove the cross-entropy loss from GAN training | GAN video-description model · ActivityNet Captions validation | METEOR: 16.49 -> 16.69 | Hurt |
| optimize with SCST rather than MLE | video-description generator · ActivityNet Captions validation | Div-1 per video: 0.55 -> 0.47 | Hurt |
| use adversarial inference rather than MLE | video-description generator · ActivityNet Captions validation | exact person word F1: 48.8 -> 49.1 | Helped |
| use adversarial inference rather than MLE | video-description generator · ActivityNet Captions validation | METEOR: 16.70 -> 16.48 | Hurt |
Closest papers by meaning
- Adversarial Semantic Alignment for Improved Image Captions (CVPR 2019 · similarity 0.88)
- Distilling Vision-Language Models on Millions of Videos (CVPR 2024 · similarity 0.87)
- Adversarial Learning of Semantic Relevance in Text to Image Synthesis (AAAI 2019 · similarity 0.87)
- Language Models with Image Descriptors are Strong Few-Shot Video-Language Learners (NeurIPS 2022 · similarity 0.87)
- Large-Scale Adversarial Training for Vision-and-Language Representation Learning (NeurIPS 2020 · similarity 0.87)
- On the Robustness of Large Multimodal Models Against Image Adversarial Attacks (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.