Research map / Multi-agent systems and games / LLM agents

Multi-Agent Debate with Memory Masking

ICLR 2026 · Multi-agent systems and games › LLM agents · approach: LLM multi-agent decision making · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MAD-M^2, a multi-agent debate variant where agents mask erroneous memories from the previous round (via self-evaluation or perplexity) before the next round, with theoretical analysis of MAD's vulnerability to wrong memories.

Problem

Multi-agent debate (MAD) lets LLM agents condition on all previous-round responses, but erroneous memories from earlier rounds can mislead agents and degrade final accuracy on reasoning tasks.

Approach

Insert an evaluation-and-masking step between debate rounds: agents either self-label each memory as YES/NO/NOT SURE (subjective masking) or keep only the lowest-perplexity response (objective masking), then debate with only the retained memories; a probabilistic analysis links MAD accuracy to the count of erroneous memories.

Key results

• MAD-M^2 (O) with Qwen2.5-Math-7B-Instruct improves MATH accuracy by 9.0 points over MAD (80.2 vs 71.2) and both AIME24&25 by 6.6 points.
• With Qwen2.5-7B-Instruct, MAD-M^2 (S) gains 1.2 points on MATH (56.8 vs 55.6) and 0.6 on MMLU-Pro (43.6 vs 43.0).
• Subjective masking increases token usage by about 13%-41% over MAD, while objective masking reduces tokens by about 30% in most cases.

Surprising findings

• MAD-M^2 (O) can fall behind plain MAD on some settings, e.g., Qwen2.5-7B-Instruct on AIME25 (MAD-M^2 (S) 3.3 vs MAD 6.7).
• Increasing debate rounds does not consistently help; performance drops for DeepSeek-Math-7B-Instruct at higher round counts.

Limitations

• Experiments use only 100 sampled questions per benchmark (except AIME) with 5 seeds, so differences are often within standard deviation.
• Subjective masking adds self-evaluation tokens; the theoretical model relies on strong assumptions (e.g., exp(-alpha N_e) success probability).
• QwQ-32B thinking outputs are truncated to 4096 tokens to serve as memories.

Experimental setup

Datasets: GSM8K MATH AIME24 AIME25 MMLU-Pro

Benchmarks: AIME24 AIME25 MMLU-Pro MATH GSM8K

Metrics: accuracy standard deviation over 5 seeds token consumption

Baselines: CoT CoT-SC MAD Sparse MAD

Builds on: multi-agent debate self-consistency chain-of-thought chain-of-thought

Closest papers by meaning

• Should we be going MAD? A Look at Multi-Agent Debate Strategies for LLMs (ICML 2024 · similarity 0.92)
• Debate or Vote: Which Yields Better Decisions in Multi-Agent Large Language Models? (NeurIPS 2025 · similarity 0.91)
• iMAD: Intelligent Multi-Agent Debate for Efficient and Accurate LLM Inference (AAAI 2026 · similarity 0.90)
• Key Decision-Makers in Multi-Agent Debates: Who Holds the Power? (AAAI 2026 · similarity 0.90)
• Unlocking the Power of Multi-Agent LLM for Reasoning: From Lazy Agents to Deliberation (ICLR 2026 · similarity 0.89)
• Evaluating Memory in LLM Agents via Incremental Multi-Turn Interactions (ICLR 2026 · similarity 0.89)