Research map / Multi-agent systems and games / LLM agents
PokéChamp: an Expert-level Minimax Language Agent
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces PokéChamp, a minimax tree-search agent in which an LLM (GPT-4o or Llama 3.1 8B) replaces action sampling, opponent modeling, and value estimation, and reports strong Gen 9 OU results against bots and a projected 1300-1500 Elo on the Showdown ladder.
Problem
Pokémon battles are a partially observable two-player game with a huge state space and strict time limits, where LLM agents have struggled with planning and game mechanics.
Approach
Embed LLMs as black-box modules inside minimax tree search: sampling candidate player actions, predicting opponent actions, and scoring leaf states, combined with a one-step lookahead damage-calculation world model and statistics from real player battles. No LLM training is required.
Key results
- With GPT-4o, PokéChamp wins 84% against the Abyssal bot in Gen 9 OU (Elo 1268).
- With Llama 3.1 8B, PokéChamp wins 64% against PokéLLMon (GPT-4o) in the reported comparisons.
- Projected online ladder Elo of 1300-1500, reported as top 30%-10% of human players, though about one third of games were lost to timeouts.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PokéChamp with GPT-4o instead of PokéLLMon with GPT-4o | Pokémon battling agent · Gen 9 OU vs. Abyssal | win rate (%): 40 -> 84 | Helped |
| Use PokéChamp with Llama 3.1:8b instead of PokéLLMon with GPT-4o | Pokémon battling agent · Gen 9 OU vs. Abyssal | win rate (%): 40 -> 56 | Helped |
| Use PokéChamp with GPT-4o instead of PokéLLMon with GPT-4o | Pokémon battling agent · 1v1 battle puzzles | win rate (%): 76% -> 86% | Helped |
| Use PokéChamp with GPT-4o instead of PokéLLMon with GPT-4o | Pokémon battling agent · Gen 8 Random Battles without Dynamax vs. Abyssal | win rate (%): 56 -> 70 | Helped |
| Use PokéChamp with Llama 3.1:8b instead of PokéLLMon with GPT-4o | Pokémon battling agent · Gen 8 Random Battles without Dynamax vs. Abyssal | win rate (%): 56 -> 64 | Helped |
| Use PokéChamp with GPT-4o instead of PokéLLMon with GPT-4o | Pokémon battling agent · Gen 8 Random Battles with Dynamax vs. Abyssal | win rate (%): 36 -> 56 | Helped |
| Use PokéChamp-Fast instead of standard PokéChamp | PokéChamp · Online ladder | Elo: 1300-1500 -> 1150-1310 | Hurt |
| Apply PokéChamp to human opponents, with timeout losses excluded | PokéChamp · Gen 9 OU online ladder | win rate (%): 76% | Helped |
Surprising findings
- About one third of online ladder games were lost by exceeding the turn time limit.
- A speed-optimized variant (PokéChamp-Fast) performed worse (Elo 1150-1310), which the authors attribute to LLM pretraining priors overriding updated metagame statistics.
Limitations
- Losses to turn-time timeouts are frequent under the live time constraints.
- Weak against stall and excessive-switching strategies due to limited lookahead and static opponent modeling.
- Online evaluation risks data contamination and burdens human players; authors urge limited ladder use.
Experimental setup
Datasets: Pokémon Showdown replay data (over 3 million battles)
Benchmarks: Gen 9 OU arena vs Abyssal Gen 8 Random Battles 1v1 battle puzzles action prediction on replay data Pokémon Showdown online ladder
Metrics: win rate Elo average number of turns top-k action accuracy
Baselines: PokéLLMon Abyssal bot One Step Lookahead Bot Max Power Bot Random
Builds on: PokéLLMon Abyssal bot chain-of-thought tree of thoughts
Closest papers by meaning
- The PokeAgent Challenge: Competitive and Long-Context Learning at Scale (NeurIPS 2026 · similarity 0.91)
- LMRL Gym: Benchmarks for Multi-Turn Reinforcement Learning with Language Models (ICML 2025 · similarity 0.85)
- Cooperation, Competition, and Maliciousness: LLM-Stakeholders Interactive Negotiation (NeurIPS 2024 · similarity 0.85)
- FightLadder: A Benchmark for Competitive Multi-Agent Reinforcement Learning (ICML 2024 · similarity 0.84)
- Can Large Language Models Master Complex Card Games? (NeurIPS 2025 · similarity 0.84)
- From Player to Master: Enhancing Test-Time Learning of LLM Agents via Reinforcement Learning over Memory (ICML 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.