Research map / Reinforcement learning / Offline RL
Constraints Penalized Q-learning for Safe Offline Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CPQ instead of BCQ-Lagrangian | Safe offline RL on mixed-policy Mujoco datasets · Hopper-v2, HalfCheetah-v2, and Walker2d-v2 | reward | Helped |
| Use CPQ instead of BEAR-Lagrangian | Safe offline RL on mixed-policy Mujoco datasets · Hopper-v2, HalfCheetah-v2, and Walker2d-v2 | reward | Helped |
| Use CBPL on the mixed offline datasets | Safe offline RL · Hopper-v2, HalfCheetah-v2, and Walker2d-v2 | reward | Hurt |
| Combine a cost critic with a divergence constraint | Naive safe offline RL approach · Safe offline RL with mixed safe and unsafe behavior data | reward | Hurt |
| Use only safe-policy data for behavior cloning | BC-Safe · Mujoco tasks with different constraint limits | constraint satisfaction | Mixed |
| Detect OOD policy actions and penalize their cost-critic values | CPQ · Safe offline RL on mixed-policy Mujoco datasets | reward | Helped |
| Use OOD detection that sometimes classifies in-distribution actions as OOD | CPQ · Hopper-v2, HalfCheetah-v2, and Walker2d-v2 | constraint value | Mixed |
| Vary the constraint limit after training to obtain policies for different safety requirements | CPQ · Mujoco tasks with different constraint limits | constraint satisfaction | Helped |
Closest papers by meaning
- Exclusively Penalized Q-learning for Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.93)
- C2IQL: Constraint-Conditioned Implicit Q-learning for Safe Offline Reinforcement Learning (ICML 2025 · similarity 0.93)
- Online Optimization for Offline Safe Reinforcement Learning (NeurIPS 2025 · similarity 0.93)
- Offline Safe Reinforcement Learning Using Trajectory Classification (AAAI 2025 · similarity 0.92)
- Constraint-Adaptive Policy Switching for Offline Safe Reinforcement Learning (AAAI 2025 · similarity 0.91)
- Latent Safety-Constrained Policy Approach for Safe Offline Reinforcement Learning (ICLR 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.