Research map / Video / Video-language understanding

Implicit Location-Caption Alignment via Complementary Masking for Weakly-Supervised Dense Video Captioning

AAAI 2025 · Video › Video-language understanding · approach: Dense video captioning · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes implicit location-caption alignment for weakly-supervised dense video captioning via complementary positive/negative Gaussian masking of video features, with a dual-mode (full/masked) GPT-2-based captioner.

Problem

Weakly-supervised dense video captioning must localize events without boundary annotations; existing methods rely on explicit alignment (reconstruction or matching) that requires costly event proposal procedures during training and inference.

Approach

Predict a differentiable Gaussian mask per caption event; caption the positively masked video and the negatively (inverse) masked video so that the two captions are complementary and together reconstruct the full video caption, aligning locations and captions implicitly without proposals.

Key results

• On ActivityNet Captions with CLIP features, the method reaches SODA 6.08, METEOR 8.48, CIDEr 33.42, outperforming weakly-supervised baselines such as EC-SL (CIDEr 21.21).
• Removing positive masked captioning drops CIDEr from 33.42 to 20.22, the largest ablation drop.
• On YouCook2 and ViTT, the method reaches CIDEr 13.38 and 17.28 versus PWS-DVC's 9.81 and 12.53 (reimplemented baselines).

Measured effects reported in the paper

Surprising findings

• A randomly initialized GPT-2 variant (no pretraining) still achieves SODA 6.06 and CIDEr 30.21 on ActivityNet Captions with CLIP features.

Limitations

• Gaussian mask overlap between events is only partially handled; performance declines as event overlap increases
• Results on YouCook2 and ViTT rely on reimplemented baselines
• Performance does not scale proportionally with larger GPT-2 backbone, attributed to overfitting from limited data

Experimental setup

Datasets: ActivityNet Captions ViTT YouCook2

Benchmarks: ActivityNet Captions ViTT YouCook2

Metrics: SODA METEOR CIDEr ROUGE-L BLEU-1 BLEU-2 BLEU-3 BLEU-4 R@Avg P@Avg

Baselines: DCE DVC PDVC Vid2Seq WSDEC ECG EC-SL PWS-DVC

Builds on: PDVC Vid2Seq WSDEC PWS-DVC

Closest papers by meaning

• Event-Equalized Dense Video Captioning (CVPR 2025 · similarity 0.89)
• Dense Video Object Captioning from Disjoint Supervision (ICLR 2025 · similarity 0.88)
• Do You Remember? Dense Video Captioning with Cross-Modal Memory Retrieval (CVPR 2024 · similarity 0.87)
• Exploring Temporal Event Cues for Dense Video Captioning in Cyclic Co-Learning (AAAI 2025 · similarity 0.87)
• Omnipotent Distillation with LLMs for Weakly-Supervised Natural Language Video Localization: When Divergence Meets Consistency (AAAI 2024 · similarity 0.86)
• Learning Multi-Scale Video-Text Correspondence for Weakly Supervised Temporal Article Gronding (AAAI 2024 · similarity 0.86)