Research map / Multi-agent systems and games / LLM agents
AIM: Adaptive Interaction in Multi-Agent Debate for Multimodal LLM Inference
Wei Fan, JinYi Yoon, Bo Ji
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AIM, an adaptive multi-agent debate framework for multimodal LLMs that selects which interaction modality agents use to exchange evidence, instead of fixed text- or graph-based interaction.
Problem
Multi-Agent Debate for multimodal reasoning depends on how agents interact, but existing methods use fixed text- or graph-based interaction that fails to convey fine-grained multimodal evidence, hurting accuracy and inflating token cost.
Approach
Treats the interaction modality (which modality agents use to exchange evidence) as an adaptive design choice in multi-agent debate, rather than a fixed text or graph channel.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| exchange task-dependent evidence (image RoI, audio segment, video clip) instead of text-only interaction (ablation) | multimodal MAD · GQA | accuracy: 68.9 -> 71.2 | Improved |
| combine text with task-dependent interaction (T+X) instead of X alone (ablation) | multimodal MAD · GQA | accuracy: 71.2 -> 77.6 | Improved |
| add graph modality to T+X (all three modalities) (ablation) | multimodal MAD · GQA | accuracy: 77.6 -> 77.5 | No clear effect |
| add graph modality to T+X (all three modalities) (ablation) | multimodal MAD · DCASE | accuracy: 50.9 -> 49.8 | Hurt |
| use AIM adaptive per-instance routing instead of the best fixed combination (T+G+X) on OKVQA (ablation) | multimodal MAD · OKVQA | accuracy: 93.5 -> 93.8 (tokens 9,773 -> 6,775) | Improved |
| use AIM adaptive routing instead of fixed T+G+X (ablation) | multimodal MAD · VCR | accuracy: 79.6 -> 84.2 | Improved |
| use AIM routing instead of best fixed combination (ablation) | multimodal MAD · IntentQA | accuracy: 71.2 -> 75.4 | Improved |
| route with question-type features only instead of all four feature groups (ablation) | AIM router · eight benchmarks (average) | accuracy: AIM improves average accuracy by 1.16% over question-type routing | Hurt |
| use cost-aware heuristic router only (no correctness features) (ablation) | AIM router · eight benchmarks (average) | accuracy: 2.51% lower average accuracy | Hurt |
| use any single-group adaptive router instead of fixed T+X (ablation) | multimodal MAD · eight benchmarks (average) | accuracy: +1.0% on average | Improved |
| use richer modality combinations rather than single modality | multimodal MAD · GQA | FaithScore: 0.50 (G) / 0.61 (T) -> 0.72 (AIM) | Improved |
| AIM vs T+X in FaithScore | multimodal MAD · VQA-v2 | FaithScore: 0.78 -> 0.77 | No clear effect |
Experimental setup
Metrics: accuracy token cost
Baselines: text-based multi-agent debate graph-based multi-agent debate
Builds on: multi-agent debate
Closest papers by meaning
- Should we be going MAD? A Look at Multi-Agent Debate Strategies for LLMs (ICML 2024 · similarity 0.90)
- iMAD: Intelligent Multi-Agent Debate for Efficient and Accurate LLM Inference (AAAI 2026 · similarity 0.89)
- Compose and Fuse: Revisiting the Foundational Bottlenecks in Multimodal Reasoning (ICLR 2026 · similarity 0.89)
- WorldMemArena: Evaluating Multimodal Agent Memory Through Action–World Interaction (NeurIPS 2026 · similarity 0.88)
- MMR-V: What's Left Unsaid? A Benchmark for Multimodal Deep Reasoning in Videos (ICLR 2026 · similarity 0.88)
- SMMBench: A Benchmark for Source-Distributed Multimodal Agent Memory (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
