Research map / Multi-agent systems and games / LLM agents

Context Learning for Multi-Agent Discussion

ICLR 2026 · Multi-agent systems and games › LLM agents · approach: Self-evolving language agent learning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes M2CL, which trains a T5-small context generator per agent to dynamically produce per-round context instructions for multi-agent discussion, improving consensus and accuracy across reasoning, embodied, and GUI tasks.

Problem

Multi-agent discussion (MAD) among LLMs suffers from discussion inconsistency because agents' pre-assigned, static contexts are misaligned and fail to fuse information exchanged among agents.

Approach

Learn a context generator per LLM that refines its instruction each round, trained with a round-wise activation-consistency objective and a self-adaptive dual variable alpha that balances coherence against divergence, with diverse initial contexts chosen by an activation-based reconstruction criterion.

Key results

• On Qwen-7B with 4 LLMs, M2CL reaches 92.5% on MMLU vs 74.2% for Best-of-N and 71.1% for Debate.
• Paper reports M2CL outperforms baselines by 20%-50% across datasets while adding at most about 10% runtime overhead.
• Context generators trained on one LLM transfer to other model architectures with consistent gains, per the paper's transferability experiments.

Surprising findings

• Best-of-N (single LLM sampled 32 times) outperforms several multi-agent baselines such as Debate, DyLAN and GPTSwarm on complex agentic tasks.

Limitations

• Diversity relies on the number of heterogeneous LLMs, which is computationally inefficient.
• Context pool and initialization depend on GPT-4o-generated contexts.
• Theoretical results rely on simplifying assumptions such as one-block transformer and linearized attention.

Experimental setup

Datasets: MMLU MATH GPQA HumanEval ALFWorld SciWorld GAIA PDDL

Benchmarks: MMLU MATH GPQA HumanEval ALFWorld SciWorld GAIA PDDL AndroidWorld

Metrics: accuracy runtime discrepancy intensity

Baselines: Single Best-of-N Debate DyLAN GPTSwarm MacNet

Builds on: multi-agent debate DyLAN GPTSwarm MacNet

Closest papers by meaning

• Should we be going MAD? A Look at Multi-Agent Debate Strategies for LLMs (ICML 2024 · similarity 0.87)
• Chain of Agents: Large Language Models Collaborating on Long-Context Tasks (NeurIPS 2024 · similarity 0.87)
• Learn-by-interact: A Data-Centric Framework For Self-Adaptive Agents in Realistic Environments (ICLR 2025 · similarity 0.87)
• iMAD: Intelligent Multi-Agent Debate for Efficient and Accurate LLM Inference (AAAI 2026 · similarity 0.86)
• Systematic Failures in Collective Reasoning under Distributed Information in Multi-Agent LLMs (ICML 2026 · similarity 0.86)
• Unlocking the Power of Multi-Agent LLM for Reasoning: From Lazy Agents to Deliberation (ICLR 2026 · similarity 0.86)