Research map / Video / Video-language understanding
Large-scale Pre-training for Grounded Video Caption Generation
Evangelos Kazakos, Cordelia Schmid, Josef Sivic
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| pre-train on HowToGround1M then fine-tune on iGround, versus fine-tuning only (ablation) | GROVE · iGround test (all frames) | AP50: 15.8 -> 40.0 | Improved |
| add fine-tuning on iGround after pre-training (ablation) | GROVE · iGround test (all frames) | CIDEr: 50.6 -> 83.5 | Improved |
| train GROVE on automatic annotations instead of applying the annotation pipeline directly | GROVE PT · iGround test (all frames) | CIDEr: 40.0 -> 50.6 | Improved |
| use GROVE video model instead of still-image GLaMM on center frame | GROVE PT · iGround test (center frame) | AP50: 20.8 -> 27.0 | Improved |
| add large-scale pre-training before fine-tuning on VidSTG (ablation) | GROVE · VidSTG test declarative | m_sIoU: 61.3 -> 63.7 | Improved |
| add large-scale pre-training before fine-tuning on ActivityNet-Entities (ablation) | GROVE · ActivityNet-Entities val | F1_all: 09.51 -> 13.39 | Improved |
| scale pre-training set from 1k to 1M videos (ablation) | GROVE · iGround val | CIDEr, AP50, recall | Improved |
| add spatio-temporal adapters (ablation) | GROVE · iGround val (PT+FT) | CIDEr: 88.9 -> 92.6 | Improved |
| unfreeze bounding box decoder and projection layers (ablation) | GROVE · iGround val (PT+FT) | CIDEr: 82.2 -> 92.6 | Improved |
| raise temporal objectness threshold from 0.0 (no head) to 0.5 (ablation) | GROVE · iGround val | AP50 | Improved |
| replace Stage 1 image grounded captioner with GIT+Llama-3+OWLv2 (ablation) | Automatic annotation · iGround val | average: 22.5 -> 20.9 | Hurt |
| replace SVO triplets with full captions in Stage 2 (ablation) | Automatic annotation · iGround val | average: 22.5 -> 21.5 | Hurt |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 2 of 2 on most shared results.
- TubeDETR — TubeDETR: Spatio-Temporal Video Grounding With Transformers (CVPR 2022) · better on 1 of 1 shared result
- STVGBert — STVGBert: A Visual-Linguistic Transformer Based Framework for Spatio-Temporal Video Grounding (ICCV 2021) · better on 1 of 1 shared result
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Comprehensive Visual Grounding for Video Description (AAAI 2024 · similarity 0.91)
- Dense Video Object Captioning from Disjoint Supervision (ICLR 2025 · similarity 0.90)
- VideoGrounding-DINO: Towards Open-Vocabulary Spatio-Temporal Video Grounding (CVPR 2024 · similarity 0.88)
- Exploiting Auxiliary Caption for Video Grounding (AAAI 2024 · similarity 0.88)
- DIBS: Enhancing Dense Video Captioning with Unlabeled Videos via Pseudo Boundary Enrichment and Online Refinement (CVPR 2024 · similarity 0.87)
- LVD-2M: A Long-take Video Dataset with Temporally Dense Captions (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
