AI Paper Atlas

Research map / Reinforcement learning / Offline RL

Online Decision Transformer

Qinqing Zheng, Amy Zhang, Aditya Grover

ICML 2022 · Reinforcement learning › Offline RL · approach: Offline-to-online fine-tuning · 21 citations (OpenAlex)

Read the paper

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
finetune offline-pretrained ODT with 200k online samplesODT · D4RL Gym tasks and datasetsnormalized return improvement from offline to online: 49.72Improved
finetune offline-pretrained IQL with 200k online samplesIQL · D4RL Gym tasks and datasetsnormalized return improvement from offline to online: 5.56Improved
finetune offline-pretrained ODT with 200k online samplesODT · AntMaze tasksnormalized return improvement from offline to online: 43.39Improved
finetune offline-pretrained IQL with 200k online samplesIQL · AntMaze tasksnormalized return improvement from offline to online: -5.2Hurt
use a stochastic policy rather than a deterministic DT-style policy (ablation)ODT online finetuning · Hopperreturn and training stabilityImproved
use curriculum exploration RTGs rather than a fixed scaled RTG (ablation)ODT online finetuning · Hopperevaluation returnHurt
set exploration RTG to twice rather than once the expert return (ablation)ODT · Hoppernormalized return: 95.70 -> 97.54Improved
set exploration RTG to twice rather than once the expert return (ablation)ODT · AntMaze umaze-diversenormalized return: 59.4 -> 58.19Hurt
add hindsight return relabeling to correct rollout RTG tokens (ablation)ODT online finetuning · Hopper mediumreturnImproved
initialize replay buffer with top-return offline trajectories rather than random trajectories (ablation)ODT · multiple environmentsreturnImproved
use evaluation context length 1 rather than 5 (ablation)ODT online finetuning · Ant medium-replayreturnImproved
remove positional embeddings (ablation)ODT online finetuning · Walker mediumreturnImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy