Research map / Reinforcement learning / Offline RL
Budgeting Counterfactual for Offline RL
Yao Liu, Pratik Chaudhari, Rasool Fakoor
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use BCOL (SAC) instead of CQL | SAC-based offline RL · D4RL total | normalized score: 1065.9 -> 1142.0 | Improved |
| Use BCOL (SAC) instead of CDC | SAC-based offline RL · D4RL total | normalized score: 1032.5 -> 1142.0 | Improved |
| Use BCOL (TD3) instead of TD3+BC | TD3-based offline RL · D4RL total | normalized score: 840.2 -> 873.3 | Improved |
| Use BCOL (SAC) instead of IQL | offline RL · D4RL AntMaze total | normalized score: 378.0 -> 396.0 | Improved |
| Use BCOL (SAC) instead of ARMOR | model-based offline RL · D4RL MuJoCo total | normalized score: 788.6 -> 746.0 | Hurt |
| Use BCOL (SAC) instead of CDC on MuJoCo | SAC-based offline RL · D4RL MuJoCo total | normalized score: 699.0 -> 746.0 | Improved |
| Use BCOL (TD3) instead of TD3+BC on MuJoCo | TD3-based offline RL · D4RL MuJoCo total | normalized score: 669.2 -> 685.6 | Improved |
| Remove the budget constraint (B to infinity) (ablation) | BCOL · D4RL MuJoCo and AntMaze | percent difference in score | Hurt |
| Assign B decisions randomly to the policy (no planning in training or testing) (ablation) | BCOL · D4RL | percent difference in score | Hurt |
| Select B policy steps randomly at test time (no test-time planning) (ablation) | BCOL · D4RL | percent difference in score | Hurt |
| Budget of only B=1 counterfactual step | BCOL (SAC) · D4RL MuJoCo | normalized score | No clear effect |
| Budget of 10 steps out of 1000 | BCOL (SAC) · D4RL MuJoCo | normalized score | Improved |
Closest papers by meaning
- Regularizing a Model-based Policy Stationary Distribution to Stabilize Offline Reinforcement Learning (ICML 2022 · similarity 0.90)
- A Clean Slate for Offline Reinforcement Learning (NeurIPS 2025 · similarity 0.90)
- Iteratively Refined Behavior Regularization for Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.89)
- Selective Uncertainty Propagation in Offline RL (AAAI 2025 · similarity 0.89)
- Delphic Offline Reinforcement Learning under Nonidentifiable Hidden Confounding (ICLR 2024 · similarity 0.89)
- Beyond OOD State Actions: Supported Cross-Domain Offline Reinforcement Learning (AAAI 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
