Research map / Reinforcement learning / Offline RL
Why So Pessimistic? Estimating Uncertainties for Offline RL through Ensembles, and Why Their Independence Matters
Seyed Kamyar Seyed Ghasemipour, Shixiang Shane Gu, Ofir Nachum
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use independently trained Q-functions with separate targets instead of shared pessimistic targets (ablation) | MSG · D4RL antmaze | task performance | Improved |
| use Shared-LCB targets instead of independent targets (ablation) | MSG · D4RL Gym | task performance | Hurt |
| use Shared-Min targets instead of independent targets (ablation) | MSG · D4RL Gym | task performance | Mixed |
| increase ensemble size from 4 to 64 (ablation) | MSG · D4RL Gym | task performance | No clear effect |
| increase ensemble size (ablation) | MSG · D4RL antmaze-large | task performance | Improved |
| use a small ensemble (N=4) rather than larger ensembles (ablation) | MSG · D4RL antmaze | task performance | Mixed |
| use efficient ensemble approximations instead of separate-network deep ensembles (ablation) | MSG · D4RL antmaze | task performance | Hurt |
| use Batch Ensembles among the efficient ensemble variants (ablation) | MSG · D4RL Gym | task performance | Improved |
| increase the CQL-style support regularizer weight (ablation) | MSG · narrow-distribution or chaotic-dynamics datasets | task performance | Improved |
| use shared pessimistic targets, which can make the LCB penalty positive | ensemble Q-value evaluation · offline RL toy MDP constructions | pessimistic value estimation: 221 of 1000 | Hurt |
| use MSG with deep ensembles rather than reported prior methods | D4RL antmaze · antmaze-medium and antmaze-large | task performance | Improved |
| use smaller network architectures than the baselines | MSG · RL Unplugged | task performance | Mixed |
Closest papers by meaning
- Neural Network Approximation for Pessimistic Offline Reinforcement Learning (AAAI 2024 · similarity 0.89)
- A Unified Principle of Pessimism for Offline Reinforcement Learning under Model Mismatch (NeurIPS 2024 · similarity 0.89)
- Information-Directed Pessimism for Offline Reinforcement Learning (ICML 2024 · similarity 0.89)
- Pessimism Meets Risk: Risk-Sensitive Offline Reinforcement Learning (ICML 2024 · similarity 0.88)
- Dynamic Uncertainty Estimation for Offline Reinforcement Learning (AAAI 2025 · similarity 0.88)
- SUMO: Search-Based Uncertainty Estimation for Model-Based Offline Reinforcement Learning (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
