Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Thinker: Learning to Plan and Act
Stephen Chung, Ivan Anokhin, David Krueger
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Wrap the environment with the Thinker-augmented MDP instead of training on the raw MDP | IMPALA actor-critic · Sokoban, 5e7 frames | solving rate (%): 56.7 -> 94.5 | Improved |
| Wrap the environment with the Thinker-augmented MDP instead of training on the raw MDP | IMPALA actor-critic · Atari 2600, 57 games | median human-normalized score (%): 102 -> 261 | Improved |
| Wrap the environment with the Thinker-augmented MDP instead of training on the raw MDP | IMPALA actor-critic · Atari 2600, 57 games | mean human-normalized score (%): 514 -> 1372 | Improved |
| Use Thinker-augmented IMPALA rather than Rainbow as the agent | Atari 2600 agent · Atari 2600, 57 games | median human-normalized score (%): 223 -> 261 | Improved |
| Train Thinker 5e7 frames vs prior DRC result reported in original paper | Thinker vs DRC · Sokoban, 2e7 frames | solving rate (%): 80 -> 88 | Improved |
| Thinker vs DRC (original paper) at the end of training | Thinker vs DRC · Sokoban, 5e7 frames | solving rate (%): 93 -> 95 | Improved |
| Use planning at test time with larger stage length K (policy trained with K=20) versus the base policy without planning (ablation) | Thinker policy · Sokoban test levels | solving rate | Improved |
| Vary stage length K over {1, 2, 5, 10, 20} during training and testing (ablation) | Thinker · Sokoban | solving rate | Mixed |
| Increase K from 10 to 20 (ablation) | Thinker · Sokoban | solving rate | No clear effect |
| Set K=1 (no model interaction) on Thinker-augmented MDP versus raw MDP (ablation) | Thinker · Sokoban | solving rate | Hurt |
| Increase K from 1 to 2 (ablation) | Thinker · Sokoban | solving rate | Improved |
| Replace dual network with single network (MuZero-style model) or remove state prediction loss (ablation) | Thinker world model · Sokoban | solving rate | Hurt |
Closest papers by meaning
- Interpreting Emergent Planning in Model-Free Reinforcement Learning (ICLR 2025 · similarity 0.87)
- When Can Model-Free Reinforcement Learning be Enough for Thinking? (NeurIPS 2025 · similarity 0.86)
- RIG: Synergizing Reasoning and Imagination in End-to-End Generalist Policy (ICLR 2026 · similarity 0.86)
- Continual Reinforcement Learning by Planning with Online World Models (ICML 2025 · similarity 0.86)
- Agent Planning with World Knowledge Model (NeurIPS 2024 · similarity 0.86)
- Hypothetical Minds: Scaffolding Theory of Mind for Multi-Agent Tasks with Large Language Models (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
