Research map / Vision-language and multimodal / Visual question answering
Emergent Communication in Interactive Sketch Question Answering
Zixing Lei, Yiming Zhang, Yuxin Xiong, Siheng Chen
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train on sketches instead of RGB images (MCAN) (ablation) | MCAN · VQAv2 sketch | overall accuracy: 40.86 -> 54.58 | Improved |
| sketch-based MCAN vs RGB-based MCAN on RGB data (upper bound) | MCAN · VQAv2 | overall accuracy: 64.74 -> 54.58 | Hurt |
| train Faster-RCNN on sketches instead of RGB (ablation) | Faster-RCNN · sketch-VG | mAP@50: 0.003 -> 0.034 | Improved |
| train Faster-RCNN on sketches instead of RGB (ablation) | Faster-RCNN · sketch-VG | weighted mAP@50: 0.010 -> 0.085 | Improved |
| sketch-trained detector evaluated on RGB vs RGB-trained detector | Faster-RCNN · VG RGB | mAP@50: 0.082 -> 0.034 | Hurt |
| use PraGeo (a=0.5) instead of Pragmatic (a=0) | sender · Human interpretability | CLIP distance: 0.9077 -> 0.1815 | Improved |
| use PraGeo instead of Pragmatic | sender · Human interpretability | human score (lower better): 0.998 -> 0.600 | Improved |
| use Geometric instead of PraGeo | sender · Human interpretability | CLIP distance: 0.1815 -> 0.1193 | Improved |
| use Pragmatic instead of Geometric at low drawing complexity (B < 0.3N) | sender · VQAv2 sketches | answer accuracy | Improved |
| use Geometric/other models at high complexity (B > 0.3N) | sender · VQAv2 sketches | answer accuracy | No clear effect |
| add multi-round interactive feedback (ablation) | PraGeo · VQAv2 sketches | answer accuracy | Improved |
| go from two-round to three-round interaction (ablation) | PraGeo · VQAv2 sketches | answer accuracy | No clear effect |
Closest papers by meaning
- Sketchtopia: A Dataset and Foundational Agents for Benchmarking Asynchronous Multimodal Communication with Iconic Feedback (CVPR 2025 · similarity 0.86)
- SketchAgent: Language-Driven Sequential Sketch Generation (CVPR 2025 · similarity 0.85)
- Learning Multi-Object Positional Relationships via Emergent Communication (AAAI 2024 · similarity 0.85)
- COLUMBUS: Evaluating COgnitive Lateral Understanding Through Multiple-Choice reBUSes (AAAI 2025 · similarity 0.83)
- Divide-and-Conquer: Tree-structured Strategy with Answer Distribution Estimator for Goal-Oriented Visual Dialogue (AAAI 2025 · similarity 0.83)
- SketchThinker-R1: Towards Efficient Sketch-Style Reasoning in Large Multimodal Models (ICLR 2026 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
