Research map / Multi-agent systems and games / Games and equilibria
Evaluating LLMs in Open-Source Games
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Evaluates LLMs on predicting/classifying strategy code in the Iterated Prisoner's Dilemma (new SPARC benchmark) and studies emergent strategies, deception, and approximate program equilibria in LLM-vs-LLM open-source games (IPD and Coin Game) using Kimi-K2 agents.
Problem
Whether LLMs can understand opponent program code in game-theoretic settings and whether LLM agents submitting programs in open-source games reach cooperative program equilibria rather than exploitative ones.
Approach
Introduces SPARC, a strategic code classification benchmark built from Axelrod IPD strategies with masking and obfuscation, and runs repeated open-source games where LLMs write Python strategy programs under cooperative, deceptive, or payoff-maximizing objectives, analyzing the resulting code features and replicator dynamics.
Key results
- o4-mini reached 88.0% accuracy on masked SPARC under zero-shot and 84.2% on obfuscated, the top results reported.
- DeepSeek-V3 reached 87.6% on masked SPARC with chain-of-thought prompting; GPT-4.1 reached 85.1% on unmasked and masked with CoT.
- Chain-of-thought raised accuracy for all models; e.g., Mistral Small rose from 40.2% (ZS) to 79.7% (CoT) on unmasked code.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Chain-of-Thought instead of Zero-Shot prompting | Mistral Small (24B) on SPARC Unmasked · SPARC Unmasked | accuracy: 40.2% -> 79.7% | Helped |
| Use Chain-of-Thought instead of Zero-Shot prompting | LLMs on SPARC · SPARC | accuracy | Helped |
| Mask class names and remove comments and docstrings (ablation) | LLMs on SPARC · SPARC Masked | accuracy | No clear effect |
| Obfuscate identifiers while preserving program logic (ablation) | LLMs on SPARC · SPARC Obfuscated | accuracy | No clear effect |
| Classify stochastic rather than deterministic strategies | LLMs on SPARC · SPARC | accuracy | Hurt |
| Set the agent objective to cooperative payoff maximization rather than payoff maximization | LLM agents in repeated open-source games · Coin Game and IPD | payoff | Mixed |
| Set the agent objective to deceptive payoff maximization | LLM agents in repeated open-source games · Coin Game and IPD | payoff | No clear effect |
| Use payoff-maximizing agents in evolutionary competition | Kimi-K2 agents · Coin Game | evolutionary fitness | Helped |
| Use cooperative payoff-maximizing agents | Kimi-K2 agents · IPD | evolutionary fitness | Helped |
Surprising findings
- DPM agents with explicit deceptive objectives fail to substantially outperform their opponents.
- Opponent code analysis (OSAS) peaks in the first meta-round and drops sharply; only 9.9% of later-round responses are independent development.
Limitations
- Experiments limited to two-player IPD and Coin Game with perfect transparency of source code.
- Possible training-data contamination from well-known IPD strategies like Tit-for-Tat.
- Formal verification of LLM-generated programs was not incorporated.
Experimental setup
Datasets: SPARC (239 Axelrod IPD strategies)
Benchmarks: SPARC
Metrics: classification accuracy payoff replicator dynamics flow cyclomatic complexity Halstead effort Opponent Script Access Score strategy feature frequency
Baselines: Tit-for-Tat GPT-4.1 DeepSeek-V3 o4-mini DeepSeek-R1
Builds on: program equilibrium (Tennenholtz) Axelrod library
Closest papers by meaning
- GTBench: Uncovering the Strategic Reasoning Capabilities of LLMs via Game-Theoretic Evaluations (NeurIPS 2024 · similarity 0.90)
- Cooperation, Competition, and Maliciousness: LLM-Stakeholders Interactive Negotiation (NeurIPS 2024 · similarity 0.89)
- More Capable, Less Cooperative? When LLMs Fail at Zero-Cost Collaboration (ICML 2026 · similarity 0.88)
- Shapley-Coop: Credit Assignment for Emergent Cooperation in Self-Interested LLM Agents (NeurIPS 2025 · similarity 0.88)
- MARSHAL: Incentivizing Multi-Agent Reasoning via Self-Play with Strategic LLMs (ICLR 2026 · similarity 0.88)
- Evaluating Generalization Capabilities of LLM-Based Agents in Mixed-Motive Scenarios Using Concordia (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.