Research map / Vision-language and multimodal / Human and embodied modeling
Multi-modal Dependency Tree for Video Captioning
Wentian Zhao, Xinxiao Wu, Jiebo Luo
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add dependency-tree structure instead of chain-structured sequence (ablation) | Multi-modal dependency tree captioner · ActivityNet Captions (GT proposals, C3D) | CIDEr: 42.30 -> 44.13 | Improved |
| add dependency-tree structure instead of chain (ablation) | captioner · Charades Captions | CIDEr: 23.50 -> 24.21 | Improved |
| add RL fine-tuning stage (ablation) | captioner · ActivityNet Captions (GT proposals, C3D) | CIDEr: 38.90 -> 44.13 | Improved |
| add RL fine-tuning stage (ablation) | captioner · Charades Captions | CIDEr: 22.53 -> 24.21 | Improved |
| add visual feature to tree-node embeddings (ablation) | captioner · ActivityNet Captions (GT proposals, C3D) | CIDEr: 35.10 -> 44.13 | Improved |
| add visual feature to tree-node embeddings (ablation) | captioner · Charades Captions | CIDEr: 21.87 -> 24.21 | Improved |
| use node-level reward instead of sequence-level reward (ablation) | captioner · ActivityNet Captions (GT proposals, C3D) | CIDEr: 41.39 -> 44.13 | Improved |
| use node-level reward instead of sequence-level reward (ablation) | captioner · Charades Captions | CIDEr: 23.41 -> 24.21 | Improved |
| use tree structure on complex-sentence subset (ablation) | captioner · Charades complex subset | ROUGE-L: 33.3 -> 39.8 | Improved |
| use tree structure on simple-sentence subset (ablation) | captioner · Charades simple subset | CIDEr: 28.4 -> 25.8 | Hurt |
| use BLEU-4 instead of CIDEr in the reward (ablation) | captioner · Charades Captions | CIDEr: 24.21 -> 22.95 | Hurt |
| use ROUGE-L instead of CIDEr in the reward (ablation) | captioner · Charades Captions | CIDEr: 24.21 -> 22.32 | Hurt |
Closest papers by meaning
- Fine-Grained Captioning of Long Videos through Scene Graph Consolidation (ICML 2025 · similarity 0.88)
- MAMS: Model-Agnostic Module Selection Framework for Video Captioning (AAAI 2025 · similarity 0.87)
- Distilling Vision-Language Models on Millions of Videos (CVPR 2024 · similarity 0.86)
- Video ReCap: Recursive Captioning of Hour-Long Videos (CVPR 2024 · similarity 0.86)
- VideoCon: Robust Video-Language Alignment via Contrast Captions (CVPR 2024 · similarity 0.86)
- Causal Graphical Models for Vision-Language Compositional Understanding (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
