Research map / Large language models / Agents, code and math
ChessGPT: Bridging Policy Learning and Language Modeling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| fine-tune RedPajama-3B on chess game+language corpus (ChessGPT-Base) | RedPajama-3B · BigBench State Tracking (Real Short) | accuracy (%): 23.2 -> 99.5 | Helped |
| fine-tune RedPajama-3B on chess corpus | RedPajama-3B · UCI to FEN (Real Long) | similarity (%): 3.8 -> 85.7 | Helped |
| fine-tune RedPajama-3B on chess corpus | RedPajama-3B · Checkmate in One (with suffix, ESM) | exact string match (%): 0.0 -> 71.4 | Helped |
| supervised chat fine-tuning (Base -> Chat) | ChessGPT-Base · BigBench State Tracking (Syn Long) | accuracy (%): 92.8 -> 85.1 | Hurt |
| supervised chat fine-tuning (Base -> Chat) | ChessGPT-Base · State value multi-choice (w/o { suffix) | multi-choice grade (%): 43.1 -> 52.8 | Helped |
| supervised chat fine-tuning (Base -> Chat) | ChessGPT-Base · Checkmate in One (with suffix, MC) | multi-choice grade (%): 66.1 -> 11.3 | Hurt |
| add '{' suffix to prompt (ablation) | ChessGPT-Base · State value multi-choice | multi-choice grade (%): 43.1 -> 53.7 | Helped |
| add '{' suffix to prompt (ablation) | ChessGPT-Chat · State value multi-choice | multi-choice grade (%): 52.8 -> 53.5 | No clear effect |
| add '{' suffix to prompt (ablation) | ChessGPT-Base · Chess annotation multi-choice | multi-choice grade (%): 33.2 -> 38.8 | Helped |
| add '{' suffix to prompt (ablation) | ChessGPT-Chat · Chess annotation multi-choice | multi-choice grade (%): 35.7 -> 34.7 | No clear effect |
| add instruction suffix 'Now white/black can checkmate in one' (ablation) | ChessGPT-Base · Checkmate in One (ESM) | exact string match (%): 26.5 -> 71.4 | Helped |
| add instruction suffix 'Now white/black can checkmate in one' (ablation) | ChessGPT-Chat · Checkmate in One (ESM) | exact string match (%): 59.4 -> 56.8 | Hurt |
Closest papers by meaning
- Mastering Board Games by External and Internal Planning with Language Models (ICML 2025 · similarity 0.90)
- Policy Learning from Tutorial Books via Understanding, Rehearsing and Introspecting (NeurIPS 2024 · similarity 0.90)
- Prompted Policy Search: Reinforcement Learning through Linguistic and Numerical Reasoning in LLMs (NeurIPS 2025 · similarity 0.88)
- PokerBench: Training Large Language Models to Become Professional Poker Players (AAAI 2025 · similarity 0.88)
- LMRL Gym: Benchmarks for Multi-Turn Reinforcement Learning with Language Models (ICML 2025 · similarity 0.87)
- Implicit Search via Discrete Diffusion: A Study on Chess (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.