Research map / Reinforcement learning / Offline RL
State Deviation Correction for Offline Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add state-deviation correction to the policy-constrained offline RL setup | SDC · GridWorld | task performance | Helped |
| Add state-deviation correction to offline RL | Mujoco control suite · Hopper-medium | normalized return: 58.0 -> 91.3±0.8 | Helped |
| Add state-deviation correction to offline RL | Mujoco control suite · Halfcheetah-medium-expert | normalized return: 62.4 -> 101.3±3.6 | Helped |
| Use SDC rather than the strongest listed baseline | Mujoco control suite · Halfcheetah-medium-replay | normalized return: 54.0 -> 47.3±1.0 | Hurt |
| Use SDC rather than the strongest listed baseline | Mujoco control suite · Walker2d-medium-replay | normalized return: 92.5 -> 30.3±4.7 | Hurt |
| Use SDC rather than the strongest listed baseline | Mujoco control suite · Walker2d-expert | normalized return: 153.9 -> 108.3±3.7 | Hurt |
| Use state-deviation correction instead of CQL as valuable expert samples decrease | Halfcheetah random-expert datasets · Halfcheetah-random-expert-0.5 through Halfcheetah-random-expert-0.9 | normalized return: 67.8±2.2 -> 88.4±1.3; 44.5±4.5 -> 90.9±2.9; 46.7±6.8 -> 85.2±1.7; 42.6±2.1 -> 88.4±3.2; 34.7±3.4 -> 70.2±5.7 | Helped |
| Use state-deviation correction instead of CQL as valuable expert samples decrease | Walker2d random-expert datasets · Walker2d-random-expert-0.5 through Walker2d-random-expert-0.9 | normalized return: 88.3±4.9 -> 88.8±3.9; 81.5±6.3 -> 65.9±5.0; 48.3±5.8 -> 85.5±4.7; 50.5±7.2 -> 49.8±5.6; 17.6±3.0 -> 26.8±2.1 | Mixed |
| Use state-deviation correction instead of CQL as valuable expert samples decrease | Hopper random-expert datasets · Hopper-random-expert-0.5 through Hopper-random-expert-0.9 | normalized return: 111.2±0.1 -> 112.8±0.2; 111.0±0.1 -> 111.6±0.1; 110.4±0.3 -> 112.0±0.3; 110.8±0.2 -> 112.3±0.1; 102.2 ±0.3 -> 108.5±0.2 | Helped |
| Increase the state perturbation scale β too much (ablation) | SDC · GridWorld | task performance | Hurt |
| Set the state perturbation scale β too small (ablation) | SDC · GridWorld | task performance | Hurt |
| Set the MMD constraint threshold η too large (ablation) | SDC · GridWorld | task performance | Hurt |
Closest papers by meaning
- Relaxed Stationary Distribution Correction Estimation for Improved Offline Policy Optimization (AAAI 2024 · similarity 0.90)
- Constrained Latent Action Policies for Model-Based Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.90)
- Offline Reinforcement Learning with OOD State Correction and OOD Action Suppression (NeurIPS 2024 · similarity 0.90)
- Dynamic Uncertainty Estimation for Offline Reinforcement Learning (AAAI 2025 · similarity 0.90)
- A Unified Principle of Pessimism for Offline Reinforcement Learning under Model Mismatch (NeurIPS 2024 · similarity 0.89)
- Selective Uncertainty Propagation in Offline RL (AAAI 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.