Research map / Large language models / Agents, code and math
Evaluating Language Model Agency Through Negotiations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes structured negotiation games between LM agents to evaluate LM agency, alignment (faithfulness, instruction-following) and performance in self-play and cross-play, testing six closed and open LMs.
Problem
Static LM benchmarks poorly capture agentic behavior, are vulnerable to data leakage, become obsolete quickly, and do not jointly assess alignment and performance in multi-turn, multi-agent interactions.
Approach
Use structured negotiation games (landlord/tenant style with payoff tables, distributive, compatible and integrative issues) between LM agents as co-evolving, complex, multi-turn benchmarks, with internal and external faithfulness metrics and instruction-following metrics, in both self-play and cross-play.
Key results
- Only closed-source models (gpt-4, gpt-3.5, claude-2, chat-bison, command) completed the negotiation tasks; LLaMA 2 models failed the qualifier.
- In self-play, gpt-4 had the highest faithfulness (0.91-0.92) and instruction-following (1.00) but low soft agreement (0.28); gpt-3.5 was most efficient (6.34 avg rounds).
- In cross-play, gpt-3.5 was the overall strongest negotiator, while gpt-4 ranked near the bottom in many categories despite strong cooperative agreed-payoff results.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add a second Issue to self-play games | LM agents · Negotiation games, soft agreement rate | soft agreement rate | Hurt |
| Use cross-play instead of self-play | Qualifying LM agents · Negotiation games, average soft agreement rate | soft agreement rate | Helped |
| Use transcript-style rather than dialogue-style negotiations (ablation) | gpt-3.5 · Negotiation games | agreement rate: 0.49 to 0.44 | Hurt |
| Show negotiation-round numbers in prompts (ablation) | gpt-4 · Negotiation games | agreement rate: almost 50% | Helped |
| Show negotiation-round numbers in prompts (ablation) | gpt-3.5 · Negotiation games | agreement rate | No clear effect |
| Use no past notes when generating a new note rather than past-note history (ablation) | gpt-3.5 and other swept models · Self-play negotiation games | agreement rate | Helped |
| Give message generation the most recent note instead of no notes (ablation) | gpt-3.5 and other swept models · Negotiation games | agreement rate | Helped |
| Give message generation all notes rather than only the most recent note (ablation) | gpt-3.5 and other swept models · Negotiation games | agreement rate | Hurt |
| Limit note and message length to 32 words rather than 64 or 128 (ablation) | gpt-3.5 and other swept models · Negotiation games | agreement rate | Hurt |
| Increase the note/message word limit from 64 to 128 (ablation) | gpt-3.5 and other swept models · Negotiation games | agreement rate | No clear effect |
| Remove access to the full message history (ablation) | gpt-3.5 and other swept models · Negotiation games | negotiation convergence | Hurt |
| Increase the number of Issues and add cooperative bargaining opportunities | LM agents · Multi-Issue negotiation games | normalized agreed-only payoff | No clear effect |
Surprising findings
- gpt-4, the most powerful model, underperformed in negotiation outcomes despite superior faithfulness and instruction-following.
- Cooperative bargaining games were most challenging; models struggled to exploit compatible or integrative opportunities.
Limitations
- Closed API models are costly, limiting number of runs and confidence (gpt-4 only ~15 runs).
- Results sensitive to prompts and settings; no exhaustive prompt optimization.
- Soft agreement metric relies on internal notes rather than extracted offers from free text.
Experimental setup
Datasets: generated negotiation transcripts (LAMEN)
Benchmarks: LAMEN negotiation games (single-issue and two-issue self-play and cross-play)
Metrics: soft and hard agreement rate normalized total payoff U normalized agreed payoff U* internal faithfulness external faithfulness note instruction-following message instruction-following format instruction-following average rounds
Baselines: gpt-4 gpt-3.5 claude-2 chat-bison command command-light LLaMA 2
Builds on: Bontempo & Iyengar (2008) negotiation rules chain-of-thought prompting ReAct
Closest papers by meaning
- PieArena: Ranking and Profiling Language Agents in Realistic Negotiation Scenarios (NeurIPS 2026 · similarity 0.89)
- Cooperation, Competition, and Maliciousness: LLM-Stakeholders Interactive Negotiation (NeurIPS 2024 · similarity 0.89)
- Re-evaluating Open-ended Evaluation of Large Language Models (ICLR 2025 · similarity 0.88)
- AgentBoard: An Analytical Evaluation Board of Multi-turn LLM Agents (NeurIPS 2024 · similarity 0.87)
- MINT: Evaluating LLMs in Multi-turn Interaction with Tools and Language Feedback (ICLR 2024 · similarity 0.87)
- Large Language Models Miss the Multi-agent Mark (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.