Research map / Reinforcement learning / Visual and representation RL
Supervised Pretraining Can Learn In-Context Reinforcement Learning
Jonathan Lee, Annie Xie, Aldo Pacchiano, Yash Chandak, Chelsea Finn, Ofir Nachum, Emma Brunskill
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use a random in-context dataset instead of dataset-only behavior at test time | DPT · Dark Room, 20 held-out goals | average return: 1.1 -> 61.5 | Improved |
| pretrain with PPO action labels and PPO in-context datasets (ablation) | DPT · Dark Room | return | No clear effect |
| pretrain on random in-context datasets with PPO action labels (ablation) | DPT · Dark Room | return | Hurt |
| reduce pretraining dataset to 10% of its size (ablation) | DPT · Dark Room offline | return: 10000 samples | Hurt |
| use 8 attention heads (ablation) | DPT · Dark Room offline | return | Hurt |
| vary embedding size, number of layers, attention heads (ablation) | DPT · Dark Room offline | return | No clear effect |
| mix expert data into the pretraining datasets (ablation) | DPT-Exp · Bandits with varying expert fraction | suboptimality | Mixed |
| change test reward distribution from Gaussian to Bernoulli | DPT · Bernoulli bandits | regret | No clear effect |
Closest papers by meaning
- Transformers as Decision Makers: Provable In-Context Reinforcement Learning via Supervised Pretraining (ICLR 2024 · similarity 0.95)
- In-Context Reinforcement Learning From Suboptimal Historical Data (ICML 2025 · similarity 0.92)
- In-Context Decision Transformer: Reinforcement Learning via Hierarchical Chain-of-Thought (ICML 2024 · similarity 0.91)
- From Weak Data to Strong Policy: Q-Targets Enable Provable In-Context Reinforcement Learning (NeurIPS 2026 · similarity 0.91)
- Generalization to New Sequential Decision Making Tasks with In-Context Learning (ICML 2024 · similarity 0.90)
- Transformers Provably Implement In-Context Reinforcement Learning with Policy Improvement (NeurIPS 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
