Research map / Multi-agent systems and games / LLM agents

Sparks of Cooperative Reasoning: LLMs as Strategic Hanabi Agents

ICML 2026 · Multi-agent systems and games › LLM agents · approach: LLM multi-agent decision making · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Evaluates 17 LLMs as Hanabi agents under three prompting scaffolds (Watson, Sherlock, Mycroft), releases HanabiLogs and HanabiRewards datasets, and RL-finetunes Qwen3-4B on HanabiRewards, which transfers to some out-of-domain tasks.

Problem

LLM cooperative reasoning under incomplete information is poorly measured: prior Hanabi LLM benchmarks use unstable protocols, limited player counts, and stateless decisions, and no public post-training datasets exist for cooperative LLM agents.

Approach

Systematically vary context engineering (minimal state, engine-provided deductions, and implicit multi-turn working memory) across 2-5 player Hanabi with fixed seeds, then post-train a small open model on move-level LLM-judged utilities from the resulting trajectories with GRPO.

Key results

• Best reasoning models reach about 15-18 of 25 Hanabi points on average, below specialized agents and human experts (about 18-23 for experienced players).
• RL on HanabiRewards raises Qwen3-4B Sherlock score by 7.5 points (+156%), bringing it within about 3 points of o4-mini and 4.2 points (+52%) above GPT-4.1.
• Multi-turn Mycroft state tracking drops scores by about 1.2 (o3) to 3.7 (o4-mini, Gemini 2.5 Pro) relative to engine-provided deductions.

Surprising findings

• Removing engine-provided possibility deductions slightly improved Grok-3-mini performance (14.5 to 15.4), suggesting agents did not effectively use deduction information.
• Adding Hanabi strategies and probabilistic reasoning reduced performance for all non-reasoning models except Mistral Medium 3.

Limitations

• Sherlock setting relies on engine-provided deductions rather than implicit tracking, which is not how humans play.
• Mycroft state tracking is imperfect and LLM-as-judge ratings define the RL reward, which may be noisy.
• Results are based on a limited number of seeds (10 games per player count) and a small RL training budget (about 6000 samples due to compute constraints).

Experimental setup

Datasets: HanabiLogs HanabiRewards HLE-generated Hanabi games EventQA IFBench AIME 2025 Group Guessing Game

Benchmarks: Hanabi (Watson, Sherlock, Mycroft settings) Group Guessing Game EventQA IFBench AIME 2025

Metrics: Hanabi score out of 25 win rate 6-way MCQ accuracy Avg@10 Pass@10 IQM with 95% CI

Baselines: SPIN-Bench o4-mini GPT-4.1 Grok-3 Claude 3.7 Sonnet GPT-4o Gemini 2.5 Pro Qwen3-4B-Instruct-2507 (base)

Builds on: GRPO Mixture of Agents Best-of-K sampling SPIN-Bench prompting

Closest papers by meaning

• MARSHAL: Incentivizing Multi-Agent Reasoning via Self-Play with Strategic LLMs (ICLR 2026 · similarity 0.90)
• GTBench: Uncovering the Strategic Reasoning Capabilities of LLMs via Game-Theoretic Evaluations (NeurIPS 2024 · similarity 0.90)
• A Generalist Hanabi Agent (ICLR 2025 · similarity 0.88)
• BALROG: Benchmarking Agentic LLM and VLM Reasoning On Games (ICLR 2025 · similarity 0.88)
• A Logical Analysis of Hanabi (AAAI 2025 · similarity 0.88)
• MEMO: Memory-Augmented Model Context Optimization for Robust Multi-Turn Multi-Agent LLM Games (ICML 2026 · similarity 0.88)