Research map / Multi-agent systems and games / LLM agents

Epistemic Gain, Aleatoric Cost: Uncertainty Decomposition in Multi-Agent Debate for Math Reasoning

ICML 2026 · Multi-agent systems and games › LLM agents · approach: Self-evolving language agent learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Decomposes answer-level uncertainty in multi-agent debate into epistemic (JSD across agents) and aleatoric (mean entropy) parts, then trains Qwen2.5-3B and Qwen3-4B agents with UMAD, a GRPO variant adding uncertainty-weighted advantages and an epistemic-influence reward.

Problem

Multi-agent debate (MAD) improves math reasoning inconsistently, with accuracy sometimes rising alongside token entropy and large differences between homogeneous and heterogeneous agent pairs; the mechanism by which information exchange shapes individual reasoning is unclear.

Approach

Decompose answer-level total uncertainty of the agent ensemble into Jensen-Shannon epistemic uncertainty (inter-agent disagreement) and average per-agent entropy (aleatoric cost), and use token-level NLL-based advantage down-weighting plus a peer-improvement intrinsic reward inside GRPO for debate training.

Key results

• In heterogeneous debate at T=5, UMAD raises the weaker Qwen2.5-3B-Instruct agent's average accuracy from 54.8 (Zero-Shot) and 60.2 (IPPO) to 66.5.
• In homogeneous debate, UMAD gains are small (A0 average 42.0 vs 39.8 Zero-Shot at T=5).
• Sys-EU generally decreases over debate turns across all configurations, while success is associated with low or decreasing Sys-AU.

Surprising findings

• Standard IPPO agents can converge to a lazy consensus with short responses such as "I agree with the other agent's answer."
• The stronger Qwen3-4B agent is not uniformly improved by UMAD and stays close to its zero-shot performance.

Limitations

• The epistemic influence reward is an observable proxy, not a true mutual information estimate.
• Token-level negative log-probability is a practical proxy for aleatoric uncertainty rather than semantic aleatoric uncertainty.
• Gains in homogeneous settings are small and saturate.

Experimental setup

Datasets: MATH

Benchmarks: MATH500 GSM8K AMC23 AIME24 AIME25

Metrics: pass@1 accuracy total uncertainty epistemic uncertainty (JSD) aleatoric uncertainty (mean entropy) flip ratio

Baselines: Zero-Shot MAD IPPO Single-GRPO

Builds on: GRPO IPPO UCAS MAPoRL

Closest papers by meaning

• The Value of Variance: Mitigating Debate Collapse in Multi-Agent Systems via Uncertainty-Driven Policy Optimization (ICML 2026 · similarity 0.92)
• Debate or Vote: Which Yields Better Decisions in Multi-Agent Large Language Models? (NeurIPS 2025 · similarity 0.88)
• When Debate Helps: Proposal Supply and Verification-Aware Readout in Multi-Agent Reasoning (NeurIPS 2026 · similarity 0.88)
• Multi-Agent Debate with Memory Masking (ICLR 2026 · similarity 0.88)
• Should we be going MAD? A Look at Multi-Agent Debate Strategies for LLMs (ICML 2024 · similarity 0.88)
• Sequential Probabilistic Uncertainty Estimation for Parallel Multi-Agent Reasoning Systems (NeurIPS 2026 · similarity 0.88)