AI Paper Atlas

Research map / Vision-language and multimodal / Human and embodied modeling

Dialog without Dialog Data: Learning Visual Dialog Agents from VQA Data

Michael Cogswell, Jiasen Lu, Rishabh Jain, Stefan Lee, Devi Parikh, Dhruv Batra

NeurIPS 2020 · Vision-language and multimodal › Human and embodied modeling · approach: Multimodal and affective interaction modeling · 7 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Fine-tune for the image-guessing task after VQA pre-trainingOurs vs Zero-shot Transfer · VQA 2 Contrast 1 Roundaccuracy: 0.73 -> 0.82Improved
Fine-tune for multi-round and random-pool image guessingOurs vs Zero-shot Transfer · VQA 2 Contrast 5 Rounds; VQA 2 Random 5 Rounds; VQA 9 Random 9 Roundsaccuracy: 0.67 -> 0.87; 0.64 -> 0.95; 0.18 -> 0.53Improved
Adapt the pretrained agent using the proposed methodOurs vs Typical Transfer · AWA 9 Random 9 Rounds; CUB 9 Random 9 Roundsaccuracy: 0.48 -> 0.74; 0.38 -> 0.74Improved
Use a discrete rather than continuous latent intention (ablation)Ours, 10-setting average · 10-setting averageaccuracy: 0.75 -> 0.81Improved
Use a discrete rather than continuous latent intention (ablation)Ours, 10-setting average · 10-setting averagerelevance: 0.66 -> 0.89Improved
Use ELBO pre-training rather than MLE (ablation)Ours, 10-setting average · 10-setting averageperplexity: 4.27 -> 2.45Improved
Use ELBO pre-training rather than MLE (ablation)Ours, 10-setting average · 10-setting averagediversity: 4.33 -> 0.23Improved
Fine-tune the speaker instead of keeping it fixed (ablation)Ours, 10-setting average · 10-setting averageperplexity: 2.57 -> 2.54No clear effect
Fine-tune the speaker instead of keeping it fixed (ablation)Ours, 10-setting average · 10-setting averageaccuracy: 0.81 -> 0.82No clear effect
Add Stage 2.A dialog-tracking training after Stage 1 (ablation)Ours, 10-setting average · 10-setting averageaccuracy: 0.72 -> 0.80Improved
Add Stage 2.B planner fine-tuning after Stage 2.A (ablation)Ours, 10-setting average · 10-setting averageaccuracy: 0.80 -> 0.81Improved
Allow questions to adapt to task feedback rather than use a parallel frozen question source (ablation)Ours, 10-setting average · 10-setting averageaccuracy: 0.78 -> 0.81Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy