Research map / Multi-agent systems and games / LLM agents

GT-HarmBench: Benchmarking AI Safety Risks Through the Lens of Game Theory

NeurIPS 2026 · Multi-agent systems and games › LLM agents · approach: Agent safety and attack benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces GT-HarmBench, 1,535 game-theoretic scenarios grounded in MIT AI Risk Repository risks, evaluates 15 frontier models on socially optimal choice, analyzes framing/order/reasoning effects, and tests five mechanism-design interventions.

Problem

Existing AI safety benchmarks evaluate single agents, leaving multi-agent risks such as coordination failure and conflict under-measured, especially in high-stakes real-world contexts.

Approach

Map real-world AI risks from the MIT AI Risk Repository onto six canonical symmetric 2x2 games (Prisoner's Dilemma, Chicken, Stag Hunt, Battle of the Sexes, Coordination, No Conflict), generate LLM-written scenarios with explicit payoffs, and measure whether models pick welfare-maximizing actions, with mechanism-design prompts as interventions.

Key results

• Across 15 frontier models, models achieve socially optimal outcomes in only 62% of scenarios (38% failure rate).
• Prisoner's Dilemma cooperation is lowest at 44%, while Chicken reaches 80% prosocial outcomes.
• Mechanism interventions raise utilitarian accuracy by +0.13 (Contracts) to +0.18 (Mediator) over a 0.59 baseline.

Surprising findings

• Surfacing explicit payoffs shifts models toward Nash play and lowers utilitarian accuracy by about 4%.
• Mechanism interventions that improve welfare (payments, penalties, mediator) reduce Nash accuracy below baseline.

Limitations

• Restricted to symmetric 2x2 games, excluding asymmetric, sequential and n-player settings.
• Third-party advisory framing rather than first-person agentic action.
• Risk severity and scenario labels are LLM-generated and only partially validated (human study of 30 scenarios).

Experimental setup

Datasets: MIT AI Risk Repository

Benchmarks: GT-HarmBench

Metrics: utilitarian accuracy Nash accuracy Rawlsian welfare Nash social welfare

Baselines: GTBench TMGBench CoopEval GovSim

Builds on: MIT AI Risk Repository GovSim CoopEval GTBench

Closest papers by meaning

• OpenAgentSafety: A Comprehensive Framework For Evaluating Real-World AI Agent Safety (ICLR 2026 · similarity 0.86)
• Unsafer in Many Turns: Benchmarking and Defending Multi-Turn Safety Risks in Tool-Using Agents (ICML 2026 · similarity 0.86)
• From Gambits to Assurances: Game-Theoretic Integration of Safety and Learning for Interactive Robotics (AAAI 2025 · similarity 0.86)
• Building a Foundational Guardrail for General Agentic Systems via Synthetic Data (ICLR 2026 · similarity 0.86)
• Towards a Science of AI Agent Reliability (ICML 2026 · similarity 0.86)
• AgentBreeder: Mitigating the AI Safety Risks of Multi-Agent Scaffolds via Self-Improvement (NeurIPS 2025 · similarity 0.86)