Research map / Reinforcement learning / Visual and representation RL

Meta-DT: Offline Meta-RL as Conditional Sequence Modeling with World Model Disentanglement

NeurIPS 2024 · Reinforcement learning › Visual and representation RL · approach: Task-inference meta-RL · 6 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Meta-DT, a decision transformer conditioned on a pretrained context-aware world model's task representation plus a self-guided prompt from the highest-prediction-error trajectory segment, enabling few- and zero-shot offline meta-RL without expert demos at test time.

Problem

Offline meta-RL generalization to unseen tasks: existing transformer-based methods (Prompt-DT, Generalized DT) need expert demonstrations or hindsight statistics at test time, and temporal-difference context methods are less stable.

Approach

Disentangle task information from behavior policies by pretraining a context encoder (GRU over h-step history) with reward and transition decoders, inject its 16-dim task embedding into a causal transformer, and build the prompt from the test-time meta-policy's own trajectory segment that maximizes world-model prediction error, so the prompt complements the world model.

Key results

• Few-shot on Medium datasets: Meta-DT achieves best return on all six MuJoCo/2D tasks, e.g., Ant-Dir 412.00 vs 287.23 for Prompt-DT.
• Zero-shot on Medium datasets: Meta-DT drops about 5% on average from few-shot, while baselines drop about 20-35%; e.g., Cheetah-Dir 542.4 zero-shot vs 146.4 for CORRO.
• Ablations on Medium datasets show removing the context (w/o_context) hurts most, e.g., Ant-Dir 290.44 vs 412.00 for full Meta-DT.

Measured effects reported in the paper

Surprising findings

• Prompt-DT's performance degrades substantially when prompts come from medium or mixed data rather than expert demonstrations.

Limitations

• Two-phase training (pretrain world model, then transformer) rather than unified training.
• Experiments use relatively lightweight, homogeneous simulated datasets; scaling not demonstrated.
• Zero-shot results on Meta-World and some baselines show high variance in places.

Experimental setup

Datasets: Point-Robot Cheetah-Vel Cheetah-Dir Ant-Dir Hopper-Param Walker-Param Reach Sweep Door-Lock

Benchmarks: Point-Robot Cheetah-Vel Cheetah-Dir Ant-Dir Hopper-Param Walker-Param Meta-World Reach Meta-World Sweep Meta-World Door-Lock

Metrics: episodic return few-shot return zero-shot return performance drop vs few-shot

Baselines: Prompt-DT Generalized DT CORRO CSRO

Builds on: Decision Transformer Prompt-DT Generalized DT SAC

Closest papers by meaning

• AMAGO-2: Breaking the Multi-Task Barrier in Meta-Reinforcement Learning with Transformers (NeurIPS 2024 · similarity 0.91)
• Decision ConvFormer: Local Filtering in MetaFormer is Sufficient for Decision Making (ICLR 2024 · similarity 0.89)
• MetaCARD: Meta-Reinforcement Learning with Task Uncertainty Feedback via Decoupled Context-Aware Reward and Dynamics Components (AAAI 2024 · similarity 0.89)
• Generalizable Task Representation Learning for Offline Meta-Reinforcement Learning with Data Limitations (AAAI 2024 · similarity 0.89)
• Towards an Information Theoretic Framework of Context-Based Offline Meta-Reinforcement Learning (NeurIPS 2024 · similarity 0.88)
• Text-to-Decision Agent: Offline Meta-Reinforcement Learning from Natural Language Supervision (NeurIPS 2025 · similarity 0.88)