Research map / Reinforcement learning / Offline RL
Regularizing a Model-based Policy Stationary Distribution to Stabilize Offline Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Regularize the learned policy's undiscounted stationary state-action distribution toward the offline data distribution (ablation) | SDM-GAN · D4RL tested datasets | average normalized return: 22.8 -> 75.2 | Helped |
| Use a moderate mixture of real data and model rollouts (f=0.5), rather than only model rollouts or mostly rollouts (ablation) | SDM-GAN · D4RL tested datasets | average normalized return: 46.2 -> 75.2; 49 -> 75.2 | Helped |
| Increase real-data percentage from f=0.5 to f=0.8 (ablation) | SDM-GAN · D4RL tested datasets | average normalized return: 75.2 -> 63.7 | Hurt |
| Remove model rollouts by setting f=1 (ablation) | SDM-GAN · D4RL tested datasets | average normalized return: 75.2 -> 59 | Hurt |
| Reduce regularization coefficient alpha from 10 to 2.5 (ablation) | SDM-GAN · D4RL tested datasets | average normalized return: 75.2 -> 57.5 | Hurt |
| Increase regularization coefficient alpha from 10 to 25 (ablation) | SDM-GAN · D4RL tested datasets | average normalized return: 75.2 -> 67.4 | Hurt |
| Replace GAN/JSD matching with WGAN-GP matching (ablation) | SDM-GAN · D4RL tested datasets | average normalized return: 75.2 -> 39.3; 36.9; 36.9 | Hurt |
| Use an implicit conditional-GAN policy rather than deterministic or Gaussian policy generators (ablation) | Toy behavior-cloning experiment · Circle dataset | action-mode coverage | Helped |
Closest papers by meaning
- Iteratively Refined Behavior Regularization for Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.93)
- Relaxed Stationary Distribution Correction Estimation for Improved Offline Policy Optimization (AAAI 2024 · similarity 0.93)
- Learning to Trust Bellman Updates: Selective State-Adaptive Regularization for Offline RL (ICML 2025 · similarity 0.91)
- A Clean Slate for Offline Reinforcement Learning (NeurIPS 2025 · similarity 0.91)
- Adaptive Advantage-Guided Policy Regularization for Offline Reinforcement Learning (ICML 2024 · similarity 0.91)
- Forecasting in Offline Reinforcement Learning for Non-stationary Environments (NeurIPS 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.