Research map / Multi-agent systems and games / Games and equilibria
Transformers as Game Players: Provable In-context Game-playing Capabilities of Pre-trained Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows theoretically that pre-trained decoder transformers can perform in-context game playing, approximating Nash equilibrium in two-player zero-sum Markov games by exactly realizing V-learning (decentralized) and VI-ULCB (centralized), with small NeurIPS-style experiments on bandit-like games.
Problem
Whether transformers pre-trained on interaction data can perform in-context learning of Nash equilibrium in competitive multi-agent games, where prior in-context RL theory covered only single-agent settings.
Approach
Frames in-context game-playing (ICGP) for two-player zero-sum Markov games, proves generalization bounds for supervised pre-training of decentralized (one transformer per player) and centralized (joint transformer) policies, and constructs transformers that exactly realize V-learning and MWU-based VI-ULCB, yielding finite-sample NE approximation guarantees.
Key results
- Transformers pre-trained with N=20 games show gradually decaying NE gap during inference, outperforming N=10 pre-training, consistent with the sqrt(1/N) dependence.
- With sufficient pre-training (N=20), pre-trained transformer performance is similar to the context algorithms EXP3 and VI-ULCB.
- Theory: transformers exactly realize decentralized V-learning (d ~ HSA, L ~ GHL_D) and centralized MWU-version VI-ULCB, giving NE-gap bounds of order sqrt(H^5 S (A v B) log(SABT)/G) plus a pre-training term.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase pre-training games from N=10 to N=20 | Pre-trained transformers · Inference tasks in decentralized and centralized game-playing | NE gap | Helped |
| Use a pre-trained transformer with sufficient pre-training games (N=20) | In-context game-playing · Inference games in decentralized and centralized learning | NE gap | Helped |
| Replace the context algorithm with its pre-trained transformer | Decentralized EXP3 and centralized VI-ULCB game-playing · Inference games | NE gap | No clear effect |
Limitations
- Experiments are small-scale and limited to bandit (stateless) settings without state transitions.
- Theoretical results assume tabular settings, exact division realized by an assumed transformer, and require augmented pre-training data covering all states.
- Future work noted: function approximation and dependence on context-algorithm data quality are open.
Experimental setup
Datasets: offline trajectories generated by V-learning / EXP3 and VI-ULCB context algorithms on synthetic two-player zero-sum game
Benchmarks: synthetic two-player zero-sum normal-form and Markov games (H=1, H=2)
Metrics: Nash equilibrium gap
Baselines: EXP3 VI-ULCB
Builds on: V-learning VI-ULCB UCB-VI in-context reinforcement learning transformer theory (Lin et al.)
Closest papers by meaning
- Transformers Provably Implement In-Context Reinforcement Learning with Policy Improvement (NeurIPS 2026 · similarity 0.89)
- Transformers as Decision Makers: Provable In-Context Reinforcement Learning via Supervised Pretraining (ICLR 2024 · similarity 0.89)
- Transformers Can Learn Temporal Difference Methods for In-Context Reinforcement Learning (ICLR 2025 · similarity 0.88)
- Accelerate Multi-Agent Reinforcement Learning in Zero-Sum Games with Subgame Curriculum Learning (AAAI 2024 · similarity 0.86)
- Towards Provable Emergence of In-Context Reinforcement Learning (NeurIPS 2025 · similarity 0.86)
- Foundations of Multi-Agent Learning in Dynamic Environments: Where Reinforcement Learning Meets Strategic Decision-Making (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.