Research map / Reinforcement learning / Offline RL
Offline Reinforcement Learning with Fisher Divergence Critic Regularization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Parameterize the critic as log behavior density plus a regularized state-action offset | SAC offline RL · D4RL MuJoCo datasets | normalized return | Mixed |
| Add gradient penalty with λ=0.1 to the offset critic (ablation) | Fisher-BRC · D4RL offline RL datasets | normalized return | Helped |
| Increase gradient penalty to λ=1.0 (ablation) | Fisher-BRC · Expert and medium datasets | normalized return | Mixed |
| Remove gradient penalty (λ=0.0) (ablation) | Fisher-BRC · D4RL offline RL datasets | training stability | Hurt |
| Remove the proposed critic representation while retaining gradient penalty (ablation) | SAC offline RL · Offline datasets | performance | No clear effect |
| Use the alternative data-action expectation for the gradient penalty (ablation) | Fisher-BRC · Offline RL | performance | No clear effect |
| Use Fisher-BRC instead of CQL | Offline RL · D4RL MuJoCo | training time: 16.3 hours vs 1.4 hours of behavioral cloning pretraining followed by 6.2 hours of policy training | Helped |
Closest papers by meaning
- Iteratively Refined Behavior Regularization for Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.92)
- Behavior-Regularized Diffusion Policy Optimization for Offline Reinforcement Learning (ICML 2025 · similarity 0.90)
- An Implicit Trust Region Approach to Behavior Regularized Offline Reinforcement Learning (AAAI 2024 · similarity 0.89)
- Learning to Trust Bellman Updates: Selective State-Adaptive Regularization for Offline RL (ICML 2025 · similarity 0.89)
- Overestimation, Overfitting, and Plasticity in Actor-Critic: the Bitter Lesson of Reinforcement Learning (ICML 2024 · similarity 0.89)
- Adaptive Advantage-Guided Policy Regularization for Offline Reinforcement Learning (ICML 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.