Research map / Vision-language and multimodal / Visual question answering
Visual Coreference Resolution in Visual Dialog using Neural Module Networks
Satwik Kottur, Jose M. F. Moura, Devi Parikh, Dhruv Batra, Marcus Rohrbach
This paper tried a common change: Add a reconstruction / auxiliary loss
add caption auxiliary loss L_C^aux to the full model · VisDial v0.9 val (VGG) · R@1: 50.49 -> 50.24 No clear effect
What 987 accepted papers report for this change
- 882 improved
- 41 no clear effect
- 20 mixed
- 44 hurt
- OPTION: Optimal Transport–Guided Flow Matching for Incomplete and Unaligned Multi-View Clustering ICML 2026 · add reconstruction loss L_rec · Scene-15 (aligned, 70% missing) · ACC: 32.4 -> 37.5Improved
- Detached Skip-Links and $R$-Probe: Decoupling Feature Aggregation from Gradient Propagation for MLLM OCR ICML 2026 · add R-probe auxiliary loss · OCRBench · score: 714.0 -> 721.0Improved
- Rethinking Time-Series Imputation as Conditional Inference along Temporal Evolution ICML 2026 · add reconstruction loss on observed entries (joint vs missing-only supervision) · ETTh1 10% missing · Masked MAE: 0.2002 -> 0.1658Improved
See this change in detailMatched by the wording of each paper's own result, read by AI from the full text. Papers report gains more often than failures, so this shows what gets published, not the chance a change works.
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add sequential distance input to the Refer scoring network (ablation) | CorefNMN · MNIST Dialog | accuracy: 88.7 -> 99.3 | Improved |
| add sequential distance input to the attention-memory baseline | AMEM · MNIST Dialog | accuracy: 89.2 -> 96.4 | Improved |
| add memory-augmented program generator (ablation) | CorefNMN · VisDial v0.9 val (VGG) | MRR: 0.618 -> 0.636 | Improved |
| add caption auxiliary loss L_C^aux to the full model (ablation) | CorefNMN · VisDial v0.9 val (VGG) | R@1: 50.49 -> 50.24 | No clear effect |
| add caption auxiliary loss L_C^aux to the model without memory (ablation) | CorefNMN\Mem · VisDial v0.9 val (VGG) | MRR: 0.617 -> 0.618 | No clear effect |
| use ResNet-152 features instead of VGG-16 | CorefNMN · VisDial v0.9 val | R@1: 50.24 -> 50.92 | Improved |
| add caption auxiliary loss L_C^aux to the full model (generative decoder) (ablation) | CorefNMN · VisDial v0.9 val (generative decoder) | mean rank: 16.39 -> 15.69 | Improved |
| add caption auxiliary loss L_C^aux to the model without memory (generative decoder) (ablation) | CorefNMN\Mem · VisDial v0.9 val (generative decoder) | MRR: 0.537 -> 0.531 | Hurt |
| add memory-augmented program generator (generative decoder, no caption loss) (ablation) | CorefNMN\L_C^aux · VisDial v0.9 val (generative decoder) | MRR: 0.537 -> 0.533 | Hurt |
| replace NMN with CorefNMN (explicit Refer module) | NMN · VisDial v1.0 test-std | MRR: 0.588 -> 0.615 | Improved |
| replace NMN with CorefNMN (explicit Refer module) | NMN · VisDial v1.0 test-std | NDCG: 0.581 -> 0.547 | Hurt |
| replace NMN with CorefNMN (generative decoder) | NMN · VisDial v0.9 val (generative decoder) | MRR: 0.542 -> 0.535 | Hurt |
Closest papers by meaning
- Visual Reference Resolution using Attention Memory for Visual Dialog (NeurIPS 2017 · similarity 0.92)
- Visual Dialog (CVPR 2017 · similarity 0.91)
- Modality-Balanced Models for Visual Dialogue (AAAI 2020 · similarity 0.91)
- Image-Question-Answer Synergistic Network for Visual Dialog (CVPR 2019 · similarity 0.90)
- V^2Dial: Unification of Video and Visual Dialog via Multimodal Experts (CVPR 2025 · similarity 0.88)
- Structure-Aware Multimodal Sequential Learning for Visual Dialog (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
