Research map / Reinforcement learning / Visual and representation RL
XLand-100B: A Large-Scale Multi-Task Dataset for In-Context Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Releases XLand-100B, a 100B-transition dataset of PPO learning histories over ~29k XLand-MiniGrid tasks, and benchmarks Algorithm Distillation and DPT on it, finding both struggle to generalize.
Problem
In-context reinforcement learning lacks large-scale, diverse datasets of learning histories with many tasks, since existing RL datasets contain few tasks or fixed policies rather than improving learning trajectories.
Approach
Collect complete PPO learning histories (with expert-action relabeling via the final policy) across nearly 30k procedurally generated XLand-MiniGrid tasks, using multi-task pretraining followed by single-task finetuning, to create a dataset suitable for AD and DPT style in-context RL.
Key results
- XLand-100B contains about 100B transitions and 2.5B episodes over 28,876 tasks, collected with 50,000 GPU hours.
- Algorithm Distillation shows in-context improvement on unseen tasks, rising from about 0.28 to 0.4 mean return on the Trivial-20B dataset.
- DPT failed to show in-context learning even on the simplest trivial tasks in the authors' experiments.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train Algorithm Distillation on XLand-Trivial-20B | Algorithm Distillation · XLand-Trivial-20B, unseen tasks | mean return: about 0.28 to 0.4 | Helped |
| Increase the number of training goals (ablation) | Algorithm Distillation · XLand, evaluation goals | return | Helped |
| Train AD for a single epoch on the full goals dataset (ablation) | Algorithm Distillation · XLand, evaluation goals | return | Hurt |
| Add HDF5 chunking with chunk size 2048 to gzip compression (ablation) | HDF5 sampling · Compression chunk-size benchmark | throughput: 173,895 -> 768,706 | Helped |
| Provide the reached-key indicator to DPT (ablation) | DPT · Key-To-Door | return | Helped |
| Train DPT without the reached-key indicator (ablation) | DPT · Key-To-Door | return | Hurt |
| Train Decision-Pretrained Transformer on XLand datasets | Decision-Pretrained Transformer · XLand-Trivial-20B and XLand-100B | in-context learning performance | No clear effect |
| Fine-tune from pretrained checkpoints rather than train from scratch (ablation) | PPO data-collection agent · XLand-100B, hard tasks | performance | Mixed |
Surprising findings
- Despite training on tens of thousands of tasks, DPT did not exhibit in-context learning, which the authors attribute to its inability to reason in POMDPs.
- Algorithm Distillation struggles to solve training tasks with deeper rulesets, even when they are in the training set.
Limitations
- All tasks share the same observation and action spaces and binary-tree task structure, limiting domain diversity.
- All learning histories were collected on single-room grid layouts, which may limit transfer to multi-room layouts.
- Fine-tuning from pretrained checkpoints may bias histories toward high initial reward and is underexplored.
Experimental setup
Datasets: XLand-100B XLand-Trivial-20B
Benchmarks: XLand-MiniGrid medium-1m unseen tasks XLand-MiniGrid trivial-1m
Metrics: mean episodic return return across episodes (in-context improvement)
Baselines: Decision-Pretrained Transformer
Builds on: Algorithm Distillation Decision-Pretrained Transformer XLand-MiniGrid PPO Decision Transformer
Closest papers by meaning
- Towards Large-Scale In-Context Reinforcement Learning by Meta-Training in Randomized Worlds (NeurIPS 2025 · similarity 0.91)
- In-Context Decision Transformer: Reinforcement Learning via Hierarchical Chain-of-Thought (ICML 2024 · similarity 0.90)
- Zero-Shot Context Generalization in Reinforcement Learning from Few Training Contexts (NeurIPS 2025 · similarity 0.89)
- In-context Exploration-Exploitation for Reinforcement Learning (ICLR 2024 · similarity 0.89)
- In-Context Reinforcement Learning From Suboptimal Historical Data (ICML 2025 · similarity 0.89)
- Zero-shot Model-based Reinforcement Learning using Large Language Models (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.