AI Paper Atlas

Research map / Video / Video-language understanding

Weakly-Supervised Generation and Grounding of Visual Descriptions With Conditional Generative Models

Effrosyni Mavroudi, René Vidal

CVPR 2022 · Video › Video-language understanding · approach: Encoder-decoder video captioning · 5 citations (OpenAlex)

Check your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Model word-to-region alignments as latent variables and ground using the learned approximate posteriorGVD-CVAE · Flickr30k Entities valBox Acc.: 22.0 -> 33.4Improved
Ground with the full-sentence approximate posterior rather than the learned priorGVD-CVAE · Flickr30k Entities valBox Acc.: 29.6 -> 33.4Improved
Use the full sentence in the posterior rather than only words through the current word (ablation)GVD-CVAE · Flickr30k Entities valBox Acc.: 26.3 -> 32.3Improved
Add transferred object-class knowledge to the inference model (ablation)GVD-CVAE · Flickr30k Entities valBox Acc.: 25.1 -> 29.6Improved
Use clipped linear β annealing instead of ordinary β annealing (ablation)GVD-CVAE · Flickr30k Entities valBox accuracy: 26.07 -> 26.31Improved
Use PI-controller β annealing instead of the vanilla CVAE objective (ablation)GVD-CVAE · Flickr30k Entities valBox accuracy: 3.29 -> 29.27Improved
Fine-tune with CIDEr-based SCST loss (ablation)GVD-CVAE · Flickr30k Entities testCIDEr: 55.3 -> 67.6Improved
Use GVD-CVAE for grounding instead of GVDGVD-CVAE · ActivityNet Entities valAcc: 14.9 -> 23.9Improved
Use GVD-CVAE instead of the GVD baselineGVD-CVAE · YouCook2 testBox accuracy macro: 37.40 -> 38.85 ± 0.20Improved
Use GVD-CVAE instead of NAFAEGVD-CVAE · YouCook2 testBox accuracy macro: 40.71 -> 38.85 ± 0.20Hurt
Add cross-entropy word-prediction loss to the vanilla CVAE loss (ablation)GVD-CVAE · Flickr30k Entities valBox accuracy: 3.29 -> 25.22Improved
Linearly anneal β from 0 to 1 (ablation)GVD-CVAE · Flickr30k Entities valBox accuracy: 25.22 -> 26.07No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy