Research map / Vision-language and multimodal / Human and embodied modeling
The Dialog Must Go On: Improving Visual Dialog via Generative Self-Training
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply GST with synthetic dialogs from unlabeled images | VisDial generative answerer · VisDial v0.9 (val) | MRR: 56.83 -> 60.03 | Helped |
| Apply GST with synthetic dialogs from unlabeled images | VisDial generative answerer · VisDial v1.0 (val) | NDCG: 63.86 -> 65.47 | Helped |
| Apply GST with synthetic dialogs from unlabeled images | VisDial generative answerer · VisDial v1.0 (val) | MRR: 52.22 -> 53.19 | Helped |
| Add perplexity-based data selection (ablation) | Student (iter1) · VisDial v1.0 (val) | NDCG: 63.96 -> 65.06 | Helped |
| Add multimodal consistency regularization (ablation) | Student (iter1) · VisDial v1.0 (val) | NDCG: 63.71 -> 65.06 | Helped |
| Retrieve in-domain images instead of using all CC12M images (ablation) | Student (iter1) · VisDial v1.0 (val) | NDCG: 64.57 -> 65.06 | Helped |
| Continue pre-training on silver image-caption pairs (ablation) | Teacher · VisDial v1.0 (val) | NDCG: 64.50 -> 63.59 | Hurt |
| Use three GST iterations instead of one (ablation) | Student · VisDial v1.0 (val) | NDCG: 65.06 -> 65.47 | Helped |
| Train GST on silver dialogs and 1% gold data | VisDial answerer · VisDial v1.0 (val), 1% gold data | NDCG: +11.09 | Helped |
| Use three GST iterations rather than one (ablation) | Student · VisDialConv, coreference attack | NDCG: 54.34 -> 55.44 | Helped |
| Use three GST iterations rather than one (ablation) | Student · VisDial v1.0 (val), FGSM attack at epsilon 0.1 | NDCG: 37.82 -> 45.43 | Helped |
| Use the entire CC12M image set rather than in-domain retrieved images (ablation) | Student (iter1) · VisDial v1.0 (val), FGSM attack | NDCG | Hurt |
Closest papers by meaning
- Best of Both Worlds: Transferring Knowledge from Discriminative Learning to a Generative Visual Dialog Model (NeurIPS 2017 · similarity 0.91)
- Visual Dialog (CVPR 2017 · similarity 0.91)
- V^2Dial: Unification of Video and Visual Dialog via Multimodal Experts (CVPR 2025 · similarity 0.89)
- Structure-Aware Multimodal Sequential Learning for Visual Dialog (AAAI 2024 · similarity 0.89)
- Personalized Visual Instruction Tuning (ICLR 2025 · similarity 0.87)
- MMDU: A Multi-Turn Multi-Image Dialog Understanding Benchmark and Instruction-Tuning Dataset for LVLMs (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.