Research map / Reinforcement learning / Offline RL
Conservative Offline Distributional Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add conservative quantile penalty (CODAC vs no penalty) (ablation) | CODAC · HalfCheetah-Medium-Expert | return: 12.1 -> 93.6 | Helped |
| add conservative quantile penalty (CODAC vs no penalty) (ablation) | CODAC · Hopper-Medium-Expert | return: 25.8 -> 111.9 | Helped |
| add conservative quantile penalty (CODAC vs no penalty) (ablation) | CODAC · Walker2d-Medium-Expert | return: 5.9 -> 111.3 | Helped |
| add conservative quantile penalty (CODAC vs no penalty) (ablation) | CODAC · HalfCheetah-Medium-Expert | positive quantile gap (%): 4.7 -> 95.3 | Helped |
| optimize CVaR0.1 (CODAC-C) instead of expected return (CODAC-N) | CODAC · Risky PointMass | violations: 108.3 -> 0.0 | Helped |
| optimize CVaR0.1 instead of expected return | CODAC · Risky Ant | CVaR0.1 return: -847.1 -> -686.6 | Helped |
| optimize CVaR0.1 instead of expected return | CODAC · Stochastic D4RL Walker2d-Medium | mean return: 1537.3 -> 1120.8 | Hurt |
| CODAC-C vs CODAC-N on stochastic D4RL | CODAC · Stochastic D4RL | CVaR0.1 return | Mixed |
| replace imitation-learning regularization (ORAAC) with conservative penalty (CODAC) | offline distributional RL · Stochastic D4RL Hopper-Mixed | mean return: 876.3 -> 1551.2 | Helped |
| CODAC vs CQL on risk-neutral D4RL | CODAC · D4RL MuJoCo 12 datasets | normalized score | Mixed |
| use distributional conservative critic vs ORAAC for value estimation | CODAC · Walker2d-Mixed lower bound | Q-estimate vs MC return: 97.6 vs 127.1 | Helped |
Closest papers by meaning
- Risk-Averse Offline Reinforcement Learning (ICLR 2021 · similarity 0.92)
- Distributional Reinforcement Learning for Risk-Sensitive Policies (NeurIPS 2022 · similarity 0.90)
- OptiDICE: Offline Policy Optimization via Stationary Distribution Correction Estimation (ICML 2021 · similarity 0.89)
- Compositional Conservatism: A Transductive Approach in Offline Reinforcement Learning (ICLR 2024 · similarity 0.89)
- Adversarially Trained Weighted Actor-Critic for Safe Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.88)
- Bayesian Design Principles for Offline-to-Online Reinforcement Learning (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.