Research map / Large language models / Agents, code and math
How Far Are LLMs from Professional Poker Players? Revisiting Game-Theoretic Reasoning with Agentic Tool Use
Minhua Lin, Enyan Dai, Hui Liu, Xianfeng Tang, Yuliang Yan, Zhenwei Dai, Jingying Zeng, Zhiwei Zhang, Fali Wang, Hongcheng Gao, Chen Luo, Xiang Zhang and 2 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add solver tool calling with BC and RL (ToolPoker) over BC-RIRL | Qwen2.5-7B · Leduc Hold'em avg | net chip gain: -40.5 -> +6.8 | Improved |
| add solver tool calling (ToolPoker) over BC-RIRL | Qwen2.5-7B · Limit Texas Hold'em avg | net chip gain: -77.6 -> +45.0 | Improved |
| add behavior cloning stage before RIRL (ablation) | Qwen2.5-7B · Leduc Hold'em avg (Table 2) | net chip gain: -23.6 -> -9.8 | Improved |
| replace sparse outcome reward (BC-SPRL) with regret-based step reward (BC-RIRL) | Qwen2.5-7B · Leduc Hold'em avg (Table 2) | net chip gain: -72.6 -> -9.8 | Improved |
| RL fine-tuning variants vs vanilla on Kuhn | Qwen2.5-7B · Kuhn Poker avg | net chip gain: -28.8 -> +17.1 | Improved |
| scale model size from 3B to 72B | Qwen2.5 · Leduc Hold'em avg | net chip gain: -135.5 -> -9.1 | Improved |
| add BC and RIRL fine-tuning to improve factual alignment | Qwen2.5-7B · Leduc Hold'em reasoning | FA: 0.86 -> 1.06 | No clear effect |
| add BC and RIRL fine-tuning to improve heuristic reasoning | Qwen2.5-7B · Leduc Hold'em reasoning | HR: 0.95 -> 1.93 | Improved |
| add BC stage to ToolPoker (ablation) | ToolPoker · Leduc Hold'em gameplay and reasoning | chip gain, HR | Improved |
| add RL fine-tuning to ToolPoker (ablation) | ToolPoker · Leduc Hold'em gameplay and reasoning | chip gain, FA, AC | Improved |
| judge with GPT-5 instead of GPT-4.1-mini | ToolPoker · Limit Texas Hold'em reasoning | LLM-judge average: 1.94 -> 1.91 | No clear effect |
| large closed LLM exploiting non-equilibrium opponents vs CFR+ | GPT-4o · Leduc Hold'em | net chip gain: +41.5 -> -8 | Mixed |
Closest papers by meaning
- GTBench: Uncovering the Strategic Reasoning Capabilities of LLMs via Game-Theoretic Evaluations (NeurIPS 2024 · similarity 0.92)
- PokerBench: Training Large Language Models to Become Professional Poker Players (AAAI 2025 · similarity 0.91)
- Mastering Board Games by External and Internal Planning with Language Models (ICML 2025 · similarity 0.89)
- Can Large Language Models Master Complex Card Games? (NeurIPS 2025 · similarity 0.88)
- GameArena: Evaluating LLM Reasoning through Live Computer Games (ICLR 2025 · similarity 0.88)
- Improving Rationality in the Reasoning Process of Language Models through Self-playing Game (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
