Research map / Reinforcement learning / Offline RL
Waypoint Transformer: Reinforcement Learning via Supervised Learning with Intermediate Targets
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| condition the policy on generated intermediate goal waypoints instead of only the global goal (ablation) | WT on AntMaze Large Play · AntMaze Large Play | normalized score: 33.0 ± 10.3 -> 72.5 ± 2.8 | Helped |
| use learned goal waypoints instead of manually selected waypoints (ablation) | WT on AntMaze Large Play · AntMaze Large Play | normalized score: 44.5 ± 2.8 vs. 33.0 ± 10.3 | Helped |
| set waypoint horizon K to 0, removing intermediate goals (ablation) | WT on AntMaze Large Play · AntMaze Large Play | normalized score: 2.2x | Hurt |
| vary waypoint horizon K away from 30 (ablation) | WT on AntMaze Large Play · AntMaze Large Play | normalized score | Mixed |
| train the goal waypoint network to lower held-out RMSE (ablation) | WT on AntMaze Large Play · AntMaze Large Play | normalized score | No clear effect |
| predict reward-to-go waypoints rather than use a constant ARTG target (ablation) | WT on Hopper Medium Replay · Hopper Medium Replay | normalized score: 66.5 ± 15.6 -> 88.9 ± 2.4 | Helped |
| predict cumulative reward-to-go with a reward waypoint network rather than update CRTG using attained rewards (ablation) | WT on Hopper Medium Replay · Hopper Medium Replay | variance of CRTG conditioning input: nearly 5x | Helped |
| add conditioning on past actions (ablation) | WT · AntMaze Large Play, Hopper Medium Replay, and Kitchen Mixed | normalized score: 66.5 ± 5.6; 76.9 ± 9.0; 65.2 ± 2.8 vs. 72.5 ± 2.8; 88.9 ± 2.4; 70.9 ± 2.1 | Hurt |
| vary dropout probability (ablation) | WT · Hopper Medium Replay, AntMaze Large Play, and Kitchen Mixed | normalized score: 75.5 ± 8.3; 68.3 ± 5.9; 72.9 ± 0.5 (0.000) vs. 88.9 ± 2.4; 72.5 ± 2.8; 70.9 ± 2.1 (0.150) | Mixed |
| vary the number of transformer layers (ablation) | WT · Hopper Medium Replay, AntMaze Large Play, and Kitchen Mixed | normalized score: 82.1 ± 8.8; 72.1 ± 5.7; 71.6 ± 1.6 (1 layer) vs. 88.9 ± 2.4; 72.5 ± 2.8; 70.9 ± 2.1 (2 layers) | Mixed |
| use generated waypoints rather than a global goal only (ablation) | WT on AntMaze Large Play · AntMaze Large Play | successful rollouts passing beyond the stitching region: 45% -> 87% | Helped |
Closest papers by meaning
- Critic-Guided Decision Transformer for Offline Reinforcement Learning (AAAI 2024 · similarity 0.90)
- Rethinking Decision Transformer via Hierarchical Reinforcement Learning (ICML 2024 · similarity 0.90)
- Transformers as Decision Makers: Provable In-Context Reinforcement Learning via Supervised Pretraining (ICLR 2024 · similarity 0.89)
- In-Context Reinforcement Learning From Suboptimal Historical Data (ICML 2025 · similarity 0.88)
- When should we prefer Decision Transformers for Offline Reinforcement Learning? (ICLR 2024 · similarity 0.88)
- Decision ConvFormer: Local Filtering in MetaFormer is Sufficient for Decision Making (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.