AI Paper Atlas

Research map / Reinforcement learning / Offline RL

Future-conditioned Unsupervised Pretraining for Decision Transformer

Zhihui Xie, Zichuan Lin, Deheng Ye, QIANG FU, Yang Wei, Shuai Li

ICML 2023 · Reinforcement learning › Offline RL · approach: Decision Transformer sequence models · 2 citations (OpenAlex)

Read the paper

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use future trajectory embeddings to condition action prediction during pretraining rather than mask the future embedding (ablation)PDT · Gym MuJoCo medium and medium-replaydownstream normalized returnImproved
Add return-prediction-guided sampling of future embeddings during finetuning (ablation)PDT · Gym MuJoCo, especially medium-replaynormalized returnImproved
Continue updating the policy during finetuning instead of freezing all parameters except the return predictor (ablation)PDT · hopper-medium and hopper-medium-replaydownstream performanceImproved
Increase future-embedding regularization strength (ablation)PDT · medium datasetsfinetuning normalized returnMixed
Increase future-embedding regularization to beta = 1 (ablation)PDT · behavior diversitybehavior diversityHurt
Condition on different predicted-return percentiles rather than always selecting the highest-return future (ablation)PDT · Gym MuJoCo after online explorationnormalized returnMixed
Use reward-free future conditioning instead of reward-conditioned ODT pretrainingPDT versus ODT · Gym MuJoCo medium-replay datasetsfinetuning normalized returnImproved
Use reward-free future conditioning instead of reward-conditioned ODT pretrainingPDT versus ODT · halfcheetah-jumpnormalized return: -31.00 ± 49.08 -> 70.39 ± 16.56Improved
Use reward-free future conditioning instead of reward-conditioned ODT pretrainingPDT versus ODT · walker2d-jumpnormalized return: 15.81 ± 14.75 -> 68.70 ± 2.90Improved
Use reward-free future conditioning instead of reward-conditioned ODT pretrainingPDT versus ODT · Generalization tasks sumnormalized return: 101.45 -> 268.21Improved
Use reward-free future conditioning instead of reward-conditioned ODT pretrainingPDT versus ODT · halfcheetah-forward-jumpnormalized return: 87.27 ± 14.41 -> 83.80 ± 2.28Hurt
Use pretrained future-conditioned policy rather than training SAC from scratchPDT versus SAC · Gym MuJoCo taskssample efficiencyImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy