Research map / Reinforcement learning / Offline RL

Behavior regularization for offline RL

Constrain learned policies toward the dataset with divergence penalties and in-sample learning.

35 accepted papers at ICML, NeurIPS, ICLR, CVPR and AAAI (2021–2026) · an approach in Offline RL, keyword cluster “offline reinforcement · offline rl · conservative”

State History Benchmarks Tried Baselines Checklist Preprints Who Export

How it got here

Most cited paper of each year

• 2021 · Offline Reinforcement Learning with Fisher Divergence Critic Regularization (ICML 2021 · 49 citations)
• 2022 · Supported Policy Optimization for Offline Reinforcement Learning (NeurIPS 2022 · 11 citations)
• 2023 · Diffusion Policies as an Expressive Policy Class for Offline Reinforcement Learning (ICLR 2023 · 34 citations)
• 2024 · Entropy-regularized Diffusion Policy with Q-Ensembles for Offline Reinforcement Learning (NeurIPS 2024 · 7 citations)
• 2025 · Distribution-Free Uncertainty Quantification in Mechanical Ventilation Treatment: A Conformal Deep Q-Learning Framework (AAAI 2025 · 1 citation)
• 2026 · Beyond Penalization: Diffusion-based Out-of-Distribution Detection and Selective Regularization in Offline Reinforcement Learning (ICLR 2026 · 0 citations)

Earliest papers here

• Offline Reinforcement Learning with Fisher Divergence Critic Regularization (ICML 2021 · 49 citations)
• Supported Policy Optimization for Offline Reinforcement Learning (NeurIPS 2022 · 11 citations)
• DR3: Value-Based Deep Reinforcement Learning Requires Explicit Regularization (ICLR 2022 · 6 citations)

Benchmark records

The best number each year that beat every earlier one, among papers of this approach reporting their own method. Numbers are extracted by AI from each paper's text; check the paper before citing.

What has been tried

24 papers here report 213 measured changes (a component added, removed or swapped, compared with a baseline): helped 59%, no clear effect 8%, hurt 21%, mixed 13%. Papers mostly publish changes that helped, so "helped" is over-represented.

Ablations in recent papers

Baseline numbers that disagree

The same baseline on the same benchmark and metric, reported more than 5% apart by different papers here. Training budgets, data splits and reimplementations differ, so state which number you compare with.

What reviewers will expect

Datasets and baselines named in the 13 papers of this approach from 2025–2026 that have a full-text summary. Leaving out the common ones is a frequent review question.

Recent preprints

Who publishes on it

Institutions on the most papers of this approach since 2023 (of 31 papers; from OpenAlex and the papers' affiliations).

• Tianjin University · 3 papers
• Tsinghua University · 3 papers
• Nanjing University of Aeronautics and Astronautics · 2 papers
• Shanxi University · 2 papers
• Google · 2 papers
• Nanjing University · 2 papers
• State University of New York at Binghamton · 2 papers
• San Diego State University · 2 papers

Use it

Download BibTeX of 30 key papersOpen the topic in the interactive map

Get alerts. Describe your idea in scime atlas and save it (free scime account): you get a weekly email when new preprints land close to it.

Related approaches

Same keyword cluster

• Offline RL sample complexity theory (156 papers)
• Simple practical offline RL (144 papers)
• Pessimism and uncertainty in offline RL (59 papers)

Elsewhere in the topic

• Offline-to-online fine-tuning (99 papers)
• Diffusion models for offline decision making (82 papers)
• Decision Transformer sequence models (40 papers)
• Offline policy evaluation and selection (8 papers)

Benchmark numbers, ablations and baseline numbers are extracted by AI from each paper's full text and can be wrong; the paper is the authority. Data: results index of 2026-11-02, citation counts from OpenAlex (2026-10-09).