Research map / Multi-agent systems and games / LLM agents

iMAD: Intelligent Multi-Agent Debate for Efficient and Accurate LLM Inference

AAAI 2026 · Multi-agent systems and games › LLM agents · approach: LLM multi-agent decision making · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes iMAD, which uses a structured self-critique single-agent response, 41 interpretable features, and an MLP classifier trained with a FocusCal loss to selectively trigger multi-agent debate only when it is likely to fix an error.

Problem

Multi-agent debate (MAD) for LLM reasoning incurs 3-5x token overhead and can overturn correct single-agent answers, so it needs a zero-shot mechanism to decide when debate is worth triggering.

Approach

Prompt a single agent for a structured self-critique (CoT, counter-argument, confidence for both), extract 41 interpretable linguistic/semantic features, and classify with a lightweight MLP trained via FocusCal (asymmetric focal loss + confidence penalty + ECE) on PubMedQA and GQA, without test-dataset tuning.

Key results

• On MEDQA, iMAD reaches 82.0% accuracy using 1,300 tokens versus 81.9% at 4,034 tokens for MAD and 80.2% at 16,832 tokens for GroupDebate.
• iMAD reduces token usage by up to 92% and improves accuracy by up to 13.5% over baselines (abstract and conclusion).
• On GSM8K, iMAD reaches 84.8% accuracy versus 76.4% for MAD, using 1,025 tokens versus 3,446.

Measured effects reported in the paper

Surprising findings

• Confidence scores are unreliable indicators of whether debate helps: incorrect answers often receive high confidence, sometimes higher than correct ones.
• On MMLU, full-debate MAD slightly outperforms iMAD (89.5% vs 89.2%), attributed to fluent, confident wrong answers giving few hesitation cues.

Limitations

• The DOWN baseline uses a threshold of 0.8 chosen from the original paper rather than tuned for this setting.
• Classifier misses some beneficial debates on short factual questions (e.g., MMLU) where wrong answers lack hesitation cues.

Experimental setup

Datasets: PubMedQA GQA MedQA MMLU GSM8K OKVQA VQA-v2 ScienceQA

Benchmarks: MedQA MMLU GSM8K OKVQA VQA-v2 ScienceQA

Metrics: accuracy tokens per question accuracy per 100k tokens inference time

Baselines: CoT Self-Consistency MAD GroupDebate DOWN

Builds on: Multi-Agent Debate Chain-of-Thought Self-Consistency DOWN focal loss

Closest papers by meaning

• Should we be going MAD? A Look at Multi-Agent Debate Strategies for LLMs (ICML 2024 · similarity 0.92)
• Debate or Vote: Which Yields Better Decisions in Multi-Agent Large Language Models? (NeurIPS 2025 · similarity 0.90)
• Multi-Agent Debate with Memory Masking (ICLR 2026 · similarity 0.90)
• When Debate Helps: Proposal Supply and Verification-Aware Readout in Multi-Agent Reasoning (NeurIPS 2026 · similarity 0.89)
• Multi-Agent Debate for LLM Judges with Adaptive Stability Detection (NeurIPS 2025 · similarity 0.89)
• Efficient Multi-Agent Reasoning via Confidence-Guided Adaptive Debate (ICML 2026 · similarity 0.89)