Research map / Reinforcement learning / Offline RL
When does return-conditioned supervised learning work for offline reinforcement learning?
David Brandfonbrener, Alberto Bietti, Jacob Buckman, Romain Laroche, Joan Bruna
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use dense reward instead of sparse or ring-of-fire reward | RCSL (RvS, DT) · Point-mass navigation | return | Improved |
| Use ring-of-fire reward, giving very low probability of high-return trajectories | RCSL (RvS, DT) · Point-mass navigation | return | Hurt |
| Use sparse reward instead of dense reward | RCSL (RvS, DT) · Point-mass navigation | return | Hurt |
| Replace RCSL with dynamic programming (IQL) under dense reward | Point-mass dense reward · Point-mass navigation | return | Improved |
| Use TD3+BC instead of IQL on point-mass data with partial state coverage | DP methods · Point-mass navigation | return | Hurt |
| Use DP methods (TD3+BC, IQL) instead of RCSL or BC on antmaze with stochastic behavior | D4RL antmaze-umaze and antmaze-medium-play · D4RL antmaze | normalized score | Improved |
| Use RCSL instead of BC on high-quality dense-reward data with poor coverage | RvS and DT · D4RL pen-human | normalized score | Improved |
| Use TD3+BC on pen-human with only 25 trajectories | DP methods · D4RL pen-human | normalized score | Hurt |
| Use DT instead of IQL on pen-human | RCSL vs IQL · D4RL pen-human | normalized score | Improved |
| Increase dataset size from pen-human (25 trajectories) to pen-expert (5000 trajectories) | DP methods vs RCSL · D4RL pen | normalized score: 25 -> 5000 trajectories | Improved |
| Use RvS instead of %BC on stitch-easy dataset | RvS · Point-mass stitch-easy | return | Improved |
| Use DT (history-conditioned) instead of RvS on stitch-easy | DT · Point-mass stitch-easy | return | Hurt |
Closest papers by meaning
- Critic-Guided Decision Transformer for Offline Reinforcement Learning (AAAI 2024 · similarity 0.92)
- Offline Safe Reinforcement Learning Using Trajectory Classification (AAAI 2025 · similarity 0.89)
- Reinformer: Max-Return Sequence Modeling for Offline RL (ICML 2024 · similarity 0.89)
- Are Expressive Models Truly Necessary for Offline RL? (AAAI 2025 · similarity 0.88)
- Adaptive $Q$-Aid for Conditional Supervised Learning in Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.88)
- Compositional Conservatism: A Transductive Approach in Offline Reinforcement Learning (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
