AI Paper Atlas

Research map / Multi-agent systems and games / Human-AI and goal-conditioned

Thinker: Learning to Plan and Act

Stephen Chung, Ivan Anokhin, David Krueger

NeurIPS 2023 · Multi-agent systems and games › Human-AI and goal-conditioned · approach: Language-grounded reinforcement agents · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Wrap the environment with the Thinker-augmented MDP instead of training on the raw MDPIMPALA actor-critic · Sokoban, 5e7 framessolving rate (%): 56.7 -> 94.5Improved
Wrap the environment with the Thinker-augmented MDP instead of training on the raw MDPIMPALA actor-critic · Atari 2600, 57 gamesmedian human-normalized score (%): 102 -> 261Improved
Wrap the environment with the Thinker-augmented MDP instead of training on the raw MDPIMPALA actor-critic · Atari 2600, 57 gamesmean human-normalized score (%): 514 -> 1372Improved
Use Thinker-augmented IMPALA rather than Rainbow as the agentAtari 2600 agent · Atari 2600, 57 gamesmedian human-normalized score (%): 223 -> 261Improved
Train Thinker 5e7 frames vs prior DRC result reported in original paperThinker vs DRC · Sokoban, 2e7 framessolving rate (%): 80 -> 88Improved
Thinker vs DRC (original paper) at the end of trainingThinker vs DRC · Sokoban, 5e7 framessolving rate (%): 93 -> 95Improved
Use planning at test time with larger stage length K (policy trained with K=20) versus the base policy without planning (ablation)Thinker policy · Sokoban test levelssolving rateImproved
Vary stage length K over {1, 2, 5, 10, 20} during training and testing (ablation)Thinker · Sokobansolving rateMixed
Increase K from 10 to 20 (ablation)Thinker · Sokobansolving rateNo clear effect
Set K=1 (no model interaction) on Thinker-augmented MDP versus raw MDP (ablation)Thinker · Sokobansolving rateHurt
Increase K from 1 to 2 (ablation)Thinker · Sokobansolving rateImproved
Replace dual network with single network (MuZero-style model) or remove state prediction loss (ablation)Thinker world model · Sokobansolving rateHurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy