Research map / Multi-agent systems and games / LLM agents
ACC-Collab: An Actor-Critic Approach to Multi-Agent LLM Collaboration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ACC-Collab, which jointly trains an actor and a critic LLM to collaborate over multi-round deliberation using DPO on preference pairs from guided collaborative trajectories.
Problem
Multi-agent debate and deliberation with off-the-shelf LLMs treats collaboration as emergent rather than learned, so collaborative ability is limited by what models already have.
Approach
Train a two-agent actor-critic team by alternating best-response optimization, where preference pairs come from off-policy "guided collaborative trajectories" steered toward and away from the ground-truth answer and scored by one-step Monte Carlo rollouts of expected final accuracy, then optimized with DPO plus NLL.
Key results
- On Llama-3, ACC-Collab+ reaches 0.894 on BoolQ versus 0.815 for DebateGPT and 0.812 for SoM (2x).
- On Mistral, ACC-Collab+ reaches 0.601 on BBH versus 0.48 for DebateGPT, and 0.672 on MMLU versus 0.594 for SFT.
- On Gemma-2, ACC-Collab reaches 0.513 on BBH versus 0.498 for SFT, but ACC-Collab underperforms DebateGPT on MMLU (0.51 vs 0.582).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train the actor-critic team with ACC-Collab instead of using the strongest listed baseline | Llama-3-8B-Instruct · BoolQ, MMLU, BBH, SCIQ, ARC | accuracy: .815 -> .887; .654 -> .644; .552 -> .593; .932 -> .952; .879 -> .881 | Helped |
| Train the actor-critic team with ACC-Collab instead of using the strongest listed baseline | Mistral-7B-Instruct · BoolQ, MMLU, BBH, SCIQ, ARC | accuracy: .848 -> .877; .594 -> .61; .48 -> .519; .871 -> .902; .834 -> .843 | Helped |
| Train the actor-critic team with ACC-Collab instead of using the strongest listed baseline | Gemma-2-2B-Instruct · BoolQ, MMLU, BBH, SCIQ | accuracy: .812 -> .84; .582 -> .51; .498 -> .513; .914 -> .918 | Mixed |
| Add a second training round (ACC-Collab+) (ablation) | ACC-Collab with Llama-3-8B-Instruct · BoolQ, MMLU, BBH, SCIQ, ARC | accuracy: .887 -> .894; .644 -> .683; .593 -> .574; .952 -> .948; .881 -> .869 | Mixed |
| Add a second training round (ACC-Collab+) (ablation) | ACC-Collab with Mistral-7B-Instruct · BoolQ, MMLU, BBH, SCIQ, ARC | accuracy: .877 -> .893; .61 -> .672; .519 -> .601; .902 -> .905; .843 -> .856 | Helped |
| Add a second training round (ACC-Collab+) (ablation) | ACC-Collab with Gemma-2-2B-Instruct · BoolQ, MMLU, BBH, SCIQ | accuracy: .84 -> .845; .51 -> .555; .513 -> .475; .918 -> .909 | Mixed |
Surprising findings
- Trained critics become more willing to disagree with the actor, whereas untrained critics are overly agreeable.
- On Gemma-2 MMLU, ACC-Collab and ACC-Collab+ underperform DebateGPT and some baselines, and ACC-Collab+ can be worse than ACC-Collab on some datasets.
Limitations
- Experiments are mainly on question-answering tasks; generalization to other tasks is untested.
- Train and test are on the same tasks, so generalization to unseen domains is unknown.
- Only 2B, 7B and 8B models are tested; scaling to larger models is unknown.
Experimental setup
Datasets: BoolQ MMLU BBH SCIQ ARC
Benchmarks: BoolQ MMLU BBH SCIQ ARC
Metrics: accuracy percent improvement over deliberation rounds process accuracy
Baselines: SoM Persona DebateTune SFT DebateGPT
Builds on: DPO Multi-Agent Debate Society of Minds DebateGPT
Closest papers by meaning
- CollabBench: Benchmarking and Unleashing Collaborative Ability of LLMs with Diverse Players via Proactive Engagement (ICML 2026 · similarity 0.89)
- Collab: Controlled Decoding using Mixture of Agents for LLM Alignment (ICLR 2025 · similarity 0.89)
- Learning “Partner-Aware” Collaborators in Multi-Party Collaboration (NeurIPS 2025 · similarity 0.89)
- CollabLLM: From Passive Responders to Active Collaborators (ICML 2025 · similarity 0.89)
- Latent Collaboration in Multi-Agent Systems (ICML 2026 · similarity 0.88)
- Graph-of-Agents: A Graph-based Framework for Multi-Agent LLM Collaboration (ICLR 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.