Research map / Video / Video-language understanding
Augmented Partial Mutual Learning with Frame Masking for Video Captioning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add annotation-wise frame masking (ablation) | Video captioning models with GRU, LSTM, or Transformer decoders · MSVD and MSR-VTT | CIDEr: 97.1 -> 101.2; 49.2 -> 49.5 | Helped |
| Add semantic encoder to the global encoder (ablation) | Video captioning model · MSVD and MSR-VTT | CIDEr: 88.6 -> 96.0; 40.3 -> 48.3 | Helped |
| Add local encoder to global and semantic encoders (ablation) | Video captioning model · MSVD and MSR-VTT | CIDEr: 96.0 -> 97.1; 48.3 -> 49.2 | Helped |
| Use APML instead of independent decoder training (ablation) | GRU captioning decoders · MSVD | CIDEr: 101.2 -> 108.3 | Helped |
| Use mutual learning between decoders (ablation) | GRU captioning decoders · MSVD | CIDEr: 101.2 -> 103.8 | Helped |
| Add augmented consistency loss to mutual learning (ablation) | GRU captioning decoders · MSVD | CIDEr: 103.8 -> 108.3 | Helped |
| Use AutoAugment without mutual-learning loss (ablation) | Single GRU or Transformer captioning model · MSVD | CIDEr: 101.2 -> 101.7; 101.0 -> 101.5 | No clear effect |
| Use hard frame masking rather than no masking (ablation) | GRU captioning model · MSVD | CIDEr: 96.4 -> 101.2 | Helped |
| Mask more than 4 frames with hard masking (ablation) | GRU captioning model · MSVD | CIDEr: 101.2 -> 100.8; 101.2 -> 98.6 | Hurt |
| Increase decoder count from 2 to 3 or 4 (ablation) | APML · MSVD | CIDEr: 108.3 -> 108.8; 108.3 -> 109.0 | Helped |
| Increase decoder count from 2 to 5 (ablation) | APML · MSVD | CIDEr: 108.3 -> 107.9 | Hurt |
| Ensemble APML-trained models rather than use one model | Video captioning model · MSVD and MSR-VTT | CIDEr: 108.3 -> 109.5; 52.2 -> 52.7 | Helped |
Closest papers by meaning
- MAMS: Model-Agnostic Module Selection Framework for Video Captioning (AAAI 2025 · similarity 0.85)
- MA-LMM: Memory-Augmented Large Multimodal Model for Long-Term Video Understanding (CVPR 2024 · similarity 0.82)
- Fine-Grained Captioning of Long Videos through Scene Graph Consolidation (ICML 2025 · similarity 0.82)
- Extending Video Masked Autoencoders to 128 frames (NeurIPS 2024 · similarity 0.82)
- Implicit Location-Caption Alignment via Complementary Masking for Weakly-Supervised Dense Video Captioning (AAAI 2025 · similarity 0.82)
- DIBS: Enhancing Dense Video Captioning with Unlabeled Videos via Pseudo Boundary Enrichment and Online Refinement (CVPR 2024 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.