Research map / Video / Video-language understanding
Text With Knowledge Graph Augmented Transformer for Video Captioning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add TextKG's two-stream transformer with retrieved knowledge to video captioning | Video captioning · YouCookII val, paragraph-level | CIDEr: 57.2 -> 75.9 | Helped |
| Add TextKG's two-stream transformer with retrieved knowledge to video captioning | Video captioning · ActivityNet Captions ae-val, paragraph-level | CIDEr: 23.4 -> 26.6 | Helped |
| Add TextKG to video captioning using CLIP video features | MSR-VTT · MSR-VTT test, micro-level | CIDEr: 60.1 -> 60.8 | Helped |
| Add transcript features to video features (ablation) | TextKG ablation · YouCookII val, paragraph-level | CIDEr: 32.1 -> 62.5 | Helped |
| Add region features to video features and transcripts (ablation) | TextKG ablation · YouCookII val, paragraph-level | CIDEr: 62.5 -> 71.3 | Helped |
| Add general and specific knowledge graphs to video, region, and transcript features (ablation) | TextKG ablation · YouCookII val, paragraph-level | CIDEr: 71.3 -> 75.9 | Helped |
| Add knowledge graphs to video and region features without transcripts (ablation) | TextKG ablation · YouCookII val, paragraph-level | CIDEr: 45.9 -> 49.8 | Helped |
| Add knowledge selection to the method with video, region, and transcript features and both KGs (ablation) | TextKG ablation · YouCookII val, paragraph-level | CIDEr: 72.0 -> 75.9 | Helped |
| Add transcript features to video, region, both KGs, and knowledge selection (ablation) | TextKG ablation · YouCookII val, paragraph-level | CIDEr: 49.8 -> 75.9 | Helped |
| Use both general and specific knowledge graphs instead of either graph alone (ablation) | TextKG with video, region, transcript features, and knowledge selection · YouCookII val, paragraph-level | CIDEr: 73.5 and 74.8 -> 75.9 | Helped |
| Use general knowledge rather than specific knowledge when transcript quality is degraded (ablation) | TextKG · YouCookII val with masked transcripts | CIDEr | Mixed |
Closest papers by meaning
- Fine-Grained Captioning of Long Videos through Scene Graph Consolidation (ICML 2025 · similarity 0.87)
- Panda-70M: Captioning 70M Videos with Multiple Cross-Modality Teachers (CVPR 2024 · similarity 0.85)
- Distilling Vision-Language Models on Millions of Videos (CVPR 2024 · similarity 0.84)
- AuroraCap: Efficient, Performant Video Detailed Captioning and a New Benchmark (ICLR 2025 · similarity 0.84)
- TOPA: Extending Large Language Models for Video Understanding via Text-Only Pre-Alignment (NeurIPS 2024 · similarity 0.84)
- Set Prediction Guided by Semantic Concepts for Diverse Video Captioning (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.