Research map / Reinforcement learning / Offline RL
A Minimalist Approach to Offline Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add behavior-cloning regularization to the TD3 policy update (ablation) | TD3 offline · D4RL MuJoCo tasks | normalized score | Helped |
| Add state-feature normalization (ablation) | TD3+BC · D4RL MuJoCo tasks | normalized score | Helped |
| Remove state-feature normalization (ablation) | TD3+BC · D4RL MuJoCo tasks | normalized score | Hurt |
| Set α to 2 or 3 instead of the default 2.5 (ablation) | TD3+BC · D4RL tasks | normalized score | No clear effect |
| Set α to 1 or 4, favoring imitation or RL more heavily (ablation) | TD3+BC · Selected D4RL tasks | normalized score | Hurt |
| Use TD3+BC rather than Fisher-BRC | Offline RL on D4RL · D4RL total | normalized score: 974.6 ±108.3 -> 979.3 ±33.4 | No clear effect |
| Use TD3+BC rather than Fisher-BRC | Offline RL training · 1 million training steps | total training time: 2h 8m -> 39m | Helped |
| Apply state-feature normalization | CQL and Fisher-BRC · D4RL tasks | normalized score | No clear effect |
| Omit state-feature normalization | TD3+BC · D4RL AntMaze | normalized score | Helped |
| Use TD3+BC rather than Decision Transformer | Offline RL on D4RL-v2 · D4RL-v2 subset with DT results | normalized score: 672.6 ±16.6 -> 677.4 ±45.6 | No clear effect |
Closest papers by meaning
- A Clean Slate for Offline Reinforcement Learning (NeurIPS 2025 · similarity 0.95)
- Offline RL Without Off-Policy Evaluation (NeurIPS 2021 · similarity 0.94)
- OptiDICE: Offline Policy Optimization via Stationary Distribution Correction Estimation (ICML 2021 · similarity 0.94)
- MOPO: Model-based Offline Policy Optimization (NeurIPS 2020 · similarity 0.94)
- Fewer May Be Better: Enhancing Offline Reinforcement Learning with Reduced Dataset (ICLR 2025 · similarity 0.93)
- Is Value Learning Really the Main Bottleneck in Offline RL? (NeurIPS 2024 · similarity 0.92)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.