Research map / Multi-agent systems and games / LLM agents

Cooperation, Competition, and Maliciousness: LLM-Stakeholders Interactive Negotiation

NeurIPS 2024 · Multi-agent systems and games › LLM agents · approach: LLM multi-agent decision making · 15 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes scorable multi-party negotiation games (6 parties, 5 issues) as an evolving benchmark for LLM cooperation, competition, and manipulation, evaluating GPT-4, GPT-3.5, Gemini Pro, Llama2/3, and Mixtral with compromising, greedy, and adversarial variants.

Problem

LLMs' communication and decision-making abilities in multi-agent negotiation are poorly understood, especially cooperation, competition, and manipulation/safety dynamics among multiple agents with secret preferences.

Approach

Scorable multi-party role-play negotiation games with secret per-party scoring functions and acceptance thresholds, tunable in difficulty via feasible-set size, plus greedy and adversarial agent variants to study cross-agent manipulation; a structured CoT prompting framework (observation, exploration, planning) is used as the baseline agent design.

Key results

• GPT-4 with best CoT prompt (no planning step, no others'-preference step) reaches 81% 5/6-way final agreement and 33% 6-way agreement, versus 0% final success for GPT-3.5 without CoT ablations reaching only 0-20%.
• Llama3 70b comes close to GPT-4 on agreement success, calculation correctness, and not revealing scores, outperforming GPT-3.5 and Gemini Pro.
• Adversarial agents targeting a specific party lower that party's final score, and adversarial coalitions with p1 and p2 can create effective attacks.

Measured effects reported in the paper

Surprising findings

• Removing the 'infer others' preferences' step and the planning step yielded higher GPT-4 final success in some rows, with the best configuration lacking the others'-preferences step.
• GPT-3.5 failed to map greedy/adversarial incentives to corresponding deals, while GPT-4 did so plausibly.

Limitations

• Experiments use only 20 runs per configuration with temperature 0 and a single base game plus three LLM-generated games.
• Only one adversary is introduced; manipulation through private communication and moderator defenses are not studied.

Experimental setup

Benchmarks: scorable negotiation games (base game and 3 new games)

Metrics: final success (5/6-way agreement) any success own score collective score wrong deals rate score leakage ratio theory-of-mind preference matching accuracy

Baselines: GPT-3.5 Gemini Pro Llama2 13b Llama2 70b Llama3 70b Mixtral 8x7B

Builds on: chain-of-thought prompting role-play negotiation exercise

Closest papers by meaning

• Evaluating Generalization Capabilities of LLM-Based Agents in Mixed-Motive Scenarios Using Concordia (NeurIPS 2025 · similarity 0.91)
• Evaluating LLMs in Open-Source Games (NeurIPS 2025 · similarity 0.89)
• Do as We Do, Not as You Think: the Conformity of Large Language Models (ICLR 2025 · similarity 0.89)
• More Capable, Less Cooperative? When LLMs Fail at Zero-Cost Collaboration (ICML 2026 · similarity 0.89)
• How Well Can LLMs Negotiate? NegotiationArena Platform and Analysis (ICML 2024 · similarity 0.89)
• Shapley-Coop: Credit Assignment for Emergent Cooperation in Self-Interested LLM Agents (NeurIPS 2025 · similarity 0.89)