AI Paper Atlas

Research map / Vision-language and multimodal / Visual question answering

EDMA: Entropy-Driven Multimodal Answering

Emanuele Mezzi, Gertjan Burghouts, Fabio Massacci, Mengyuan Zhang

NeurIPS 2026 · Vision-language and multimodal › Visual question answering · approach: Open-domain multimodal QA · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes EDMA, an entropy-driven approach to multimodal question answering that quantifies how much each modality contributes to the final answer.

Problem

Multimodal question answering must integrate heterogeneous sources, select relevant modalities, and ignore distractors; existing MLLM-based methods split the task into intermediate steps but do not quantify each modality's contribution to the final answer.

Approach

Quantify the effect of each modality on reaching the final answer using entropy, and use this entropy signal to drive modality selection and answering, rather than relying on unquantified intermediate steps.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use EDMA entropy-driven answering instead of Planning Prompting6 MLLMs · MultiModalQA (val)EM / F1 / LLM-J: +22.7 EM, +14.0 F1, +8.5 LLM-JImproved
use EDMA instead of Planning Prompting6 MLLMs · ManyModalQAEM / F1 / LLM-J: +23.0 EM, +11.5 F1, +0.6 LLM-JImproved
select trajectory by minimum logical entropy instead of uniformly at random (ablation)EDMA · MultiModalQA (val), 6 MLLMsLLM-J: 47.2 -> 79.3Improved
add cross-modal conditioning (multimodal partitions) on top of unimodal partitions (ablation)EDMA · MultiModalQA (val)LLM-J: 65.5 -> 80.5 (GPT-5.2), 48.0 -> 65.9 (Nova-2-Lite)Improved
use EDMA instead of self-consistency (N=6) baselines at matched costCoT-SC and PP-SC · MultiModalQA (val), 4 MLLMsLLM-J: 71.2 / 70.8 -> 78.2Improved
use EDMA instead of multi-agent MAMMQAMAMMQA · MultiModalQA (val), 4 MLLMsLLM-J: 75.7 -> 78.2 average; Kimi-K2.5 88.4 vs 83.2Mixed
apply self-consistency (N=6) to Planning Prompting on larger backbones (ablation)PP · MultiModalQA (val), Claude-4.6EM: -4.9 EM, -3.2 F1, -3.1 LLM-JHurt
apply self-consistency (N=6) to Planning Prompting on Nova-2-Lite (ablation)PP · MultiModalQA (val), Nova-2-LiteLLM-J: +5.5 EM, +9.5 F1, +17.8 LLM-JImproved
choose lowest-entropy trajectory instead of median or highest entropy (ablation)EDMA · MultiModalQA, all target modalitiesfalse discovery rateImproved
apply EDMA to cross-dataset ManyModalQA versus prior specialized systemsUniMMQA · ManyModalQAEM: 52.1 -> 88.0Improved
use EDMA instead of single-pass prompting in compute termsEDMA · compute costcost ratio vs CoT: 5.91x cost, 22.83x latency vs CoTHurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy