Research map / Vision-language and multimodal / Visual question answering
Visual Reasoning with Multi-hop Feature Modulation
Florian Strub, Mathieu Seurin, Ethan Perez, Harm de Vries, Jeremie Mary, Philippe Preux, Aaron CourvilleOlivier Pietquin
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use attention over the dialogue to update the context (instead of a shared MLP update) (ablation) | Multi-hop FiLM · GuessWhat?! Guesser | test error: 37.3 -> 30.5 | Improved |
| replace single-hop FiLM generator with multi-hop | FiLM model · GuessWhat?! Guesser | test error: 35.6% -> 30.5% | Improved |
| replace single-hop FiLM generator with multi-hop | FiLM model · GuessWhat?! Oracle | test error: 17.6% -> 16.9% | Improved |
| replace single-hop FiLM generator with multi-hop | FiLM model · RefCOCO+ val | accuracy: 72.1% -> 73.4% | Improved |
| add visual feedback to the context embedding (ablation) | Multi-hop FiLM · ReferIt (RefCOCO, RefCOCO+, RefCOCOg, RefClef) | accuracy: 83.5% -> 84.9% (RefCOCO val) | Improved |
| add visual feedback to the context embedding (ablation) | Multi-hop FiLM · GuessWhat?! Oracle | test error: 16.9% -> 17.1% | No clear effect |
| add visual feedback to the context embedding (ablation) | Multi-hop FiLM · GuessWhat?! Guesser (crop+img) | test error: 35.3% -> 34.3% (pointing task crop) | No clear effect |
| add object category label (ablation) | Multi-hop FiLM · GuessWhat?! Guesser (crop+img) | test error: 33.4% -> 30.5% | Improved |
| add crop input on top of image input (ablation) | Multi-hop FiLM · GuessWhat?! Guesser | test error: 35.0% -> 30.5% | Improved |
| add image and crop inputs to a baseline using only object category and spatial location (ablation) | Baseline NN + MLB Oracle · GuessWhat?! Oracle | test error: 20.6% -> 21.1% | Hurt |
| add previous question-answer pairs (dialogue) to the question input (ablation) | Baseline NN + MLB Oracle · GuessWhat?! Oracle | test error: 21.8% -> 21.1% | Improved |
| add crop and image together to the baseline Guesser (ablation) | Baseline NN Guesser · GuessWhat?! Guesser | test error: 38.3% -> 45.1% | Hurt |
Closest papers by meaning
- FiLM: Visual Reasoning with a General Conditioning Layer (AAAI 2018 · similarity 0.90)
- HopChain: Multi-Hop Data Synthesis for Generalizable Vision-Language Reasoning (NeurIPS 2026 · similarity 0.87)
- MoDA: Modulation Adapter for Fine-Grained Visual Understanding in Instructional MLLMs (ICML 2026 · similarity 0.86)
- CRIT: Graph-Based Automatic Data Synthesis to Enhance Cross-Modal Multi-Hop Reasoning (CVPR 2026 · similarity 0.86)
- Visual CoT: Advancing Multi-Modal Language Models with a Comprehensive Dataset and Benchmark for Chain-of-Thought Reasoning (NeurIPS 2024 · similarity 0.85)
- Grounded Multi-Hop VideoQA in Long-Form Egocentric Videos (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
