Research map / Video / Video-language understanding
End-to-End Dense Video Captioning With Masked Transformer
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace Bi-LSTM context encoding with self-attention in the proposal model (ablation) | ActivityNet event proposal model · ActivityNet Captions validation | Average Recall (%): 50.65 -> 52.95 | Helped |
| Improve the proposal network with deeper temporal convolutions and more balanced anchor candidates (ablation) | ProcNets-prop · ActivityNet Captions validation | Average Recall (%): 47.01 -> 50.65 | Helped |
| Use end-to-end differentiable masking to let captioning influence proposal predictions (ablation) | Masked Transformer · ActivityNet Captions validation, learned proposals | METEOR: 9.43 -> 9.56 | Helped |
| Replace Bi-LSTM+TempoAttn with the end-to-end Masked Transformer | ActivityNet Captions dense captioning · ActivityNet Captions validation, learned proposals | METEOR: 7.49 -> 9.56 | Helped |
| Replace Bi-LSTM+TempoAttn with the proposed model | YouCookII dense captioning · YouCookII validation, learned proposals | METEOR: 4.62 -> 6.58 | Helped |
| Replace Bi-LSTM+TempoAttn with the proposed model | YouCookII captioning · YouCookII validation, GT proposals | METEOR: 8.15 -> 11.20 | Helped |
| Increase the Transformer from 1 to 2 layers (ablation) | Masked Transformer captioner · ActivityNet Captions validation, GT proposals | METEOR: 10.92 -> 11.06 | Helped |
| Increase the Transformer from 2 to 4 layers (ablation) | Masked Transformer captioner · ActivityNet Captions validation, GT proposals | METEOR: 11.06 -> 11.11 | Helped |
| Increase the Transformer from 4 to 6 layers (ablation) | Masked Transformer captioner · ActivityNet Captions validation, GT proposals | METEOR: 11.11 -> 11.10 | No clear effect |
| Replace Bi-LSTM+TempoAttn with self-attention-based captioning | ActivityNet captions of long events · ActivityNet Captions, long events, GT proposals | METEOR: 5.39 -> 5.90 | Helped |
| Replace Bi-LSTM+TempoAttn with self-attention-based dense captioning | ActivityNet captions of long events · ActivityNet Captions, long events, learned proposals | METEOR: 3.99 -> 5.93 | Helped |
| Replace Bi-LSTM+TempoAttn with self-attention-based captioning | ActivityNet captions of short events · ActivityNet Captions, short events, learned proposals | METEOR: 4.43 -> 5.06 | Helped |
Closest papers by meaning
- Event-Equalized Dense Video Captioning (CVPR 2025 · similarity 0.90)
- Streaming Dense Video Captioning (CVPR 2024 · similarity 0.90)
- Do You Remember? Dense Video Captioning with Cross-Modal Memory Retrieval (CVPR 2024 · similarity 0.89)
- Implicit Location-Caption Alignment via Complementary Masking for Weakly-Supervised Dense Video Captioning (AAAI 2025 · similarity 0.89)
- HiCM²: Hierarchical Compact Memory Modeling for Dense Video Captioning (AAAI 2025 · similarity 0.87)
- DIBS: Enhancing Dense Video Captioning with Unlabeled Videos via Pseudo Boundary Enrichment and Online Refinement (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.