Research map / Video / Video-language understanding
Weakly-Supervised Generation and Grounding of Visual Descriptions With Conditional Generative Models
Effrosyni Mavroudi, René Vidal
Check your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Model word-to-region alignments as latent variables and ground using the learned approximate posterior | GVD-CVAE · Flickr30k Entities val | Box Acc.: 22.0 -> 33.4 | Improved |
| Ground with the full-sentence approximate posterior rather than the learned prior | GVD-CVAE · Flickr30k Entities val | Box Acc.: 29.6 -> 33.4 | Improved |
| Use the full sentence in the posterior rather than only words through the current word (ablation) | GVD-CVAE · Flickr30k Entities val | Box Acc.: 26.3 -> 32.3 | Improved |
| Add transferred object-class knowledge to the inference model (ablation) | GVD-CVAE · Flickr30k Entities val | Box Acc.: 25.1 -> 29.6 | Improved |
| Use clipped linear β annealing instead of ordinary β annealing (ablation) | GVD-CVAE · Flickr30k Entities val | Box accuracy: 26.07 -> 26.31 | Improved |
| Use PI-controller β annealing instead of the vanilla CVAE objective (ablation) | GVD-CVAE · Flickr30k Entities val | Box accuracy: 3.29 -> 29.27 | Improved |
| Fine-tune with CIDEr-based SCST loss (ablation) | GVD-CVAE · Flickr30k Entities test | CIDEr: 55.3 -> 67.6 | Improved |
| Use GVD-CVAE for grounding instead of GVD | GVD-CVAE · ActivityNet Entities val | Acc: 14.9 -> 23.9 | Improved |
| Use GVD-CVAE instead of the GVD baseline | GVD-CVAE · YouCook2 test | Box accuracy macro: 37.40 -> 38.85 ± 0.20 | Improved |
| Use GVD-CVAE instead of NAFAE | GVD-CVAE · YouCook2 test | Box accuracy macro: 40.71 -> 38.85 ± 0.20 | Hurt |
| Add cross-entropy word-prediction loss to the vanilla CVAE loss (ablation) | GVD-CVAE · Flickr30k Entities val | Box accuracy: 3.29 -> 25.22 | Improved |
| Linearly anneal β from 0 to 1 (ablation) | GVD-CVAE · Flickr30k Entities val | Box accuracy: 25.22 -> 26.07 | No clear effect |
Closest papers by meaning
- Learning to Generate Grounded Visual Captions without Localization Supervision (ECCV 2020 · similarity 0.92)
- Weakly-Supervised Visual Grounding of Phrases With Linguistic Structures (CVPR 2017 · similarity 0.91)
- Your Large Vision-Language Model Only Needs A Few Attention Heads For Visual Grounding (CVPR 2025 · similarity 0.89)
- Comprehensive Visual Grounding for Video Description (AAAI 2024 · similarity 0.89)
- Cycle-Consistency Learning for Captioning and Grounding (AAAI 2024 · similarity 0.89)
- Connecting the Dots: Training-Free Visual Grounding via Agentic Reasoning (AAAI 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
