Research map / Reinforcement learning / Offline RL
You Can’t Count on Luck: Why Decision Transformers and RvS Fail in Stochastic Environments
Keiran Paster, Sheila A. McIlraith, Jimmy Ba
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace conditioning on sampled trajectory returns with learned cluster-average returns | RvS / Decision Transformer · Gambling, Connect Four, and 2048 | alignment between target and achieved return | Improved |
| Replace conditioning on sampled trajectory returns with learned cluster-average returns | RvS / Decision Transformer · Connect Four | average return: 0.2 -> win and lose with perfect accuracy | Improved |
| Replace conditioning on sampled trajectory returns with learned cluster-average returns | RvS / Decision Transformer · 2048 | win rate: about $60\%$ -> $30\%$ to $80\%$ | Improved |
| Replace conditioning on sampled trajectory returns with learned cluster-average returns | RvS / Decision Transformer · Gambling | maximum expected return: cannot get positive reward -> 1.0 | Improved |
| Train on more offline data | Return-conditioned RvS · 2048 | alignment between target and achieved return: 5\% -> 100\% | No clear effect |
| Train on more offline data | ESPER · 2048 | alignment between target and achieved return: 5\% -> 100\% | Improved |
| Use trajectory statistics that allow the dynamics model to exploit stochastic outcomes (ablation) | ESPER clustering and policy · Gambling | agent performance | Hurt |
| Use an MLP policy instead of a transformer policy (ablation) | ESPER · Stochastic benchmark tasks | performance | No clear effect |
| Use ESPER rather than return-conditioned Decision Transformer (ablation) | RvS · D4RL Mujoco tasks | performance | No clear effect |
Closest papers by meaning
- Optimal Dynamic Regret by Transformers for Non-Stationary Reinforcement Learning (NeurIPS 2025 · similarity 0.87)
- Critic-Guided Decision Transformer for Offline Reinforcement Learning (AAAI 2024 · similarity 0.87)
- The Effective Horizon Explains Deep RL Performance in Stochastic Environments (ICLR 2024 · similarity 0.86)
- Beyond Expected Return: Accounting for Policy Reproducibility When Evaluating Reinforcement Learning Algorithms (AAAI 2024 · similarity 0.86)
- ACT: Empowering Decision Transformer with Dynamic Programming via Advantage Conditioning (AAAI 2024 · similarity 0.86)
- A Provable Approach for End-to-End Safe Reinforcement Learning (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
