Research map / Reinforcement learning / Offline RL
Optimal Conservative Offline RL with General Function Approximation via Augmented Lagrangian
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add augmented Lagrangian occupancy-validity penalties to MIS | CORAL (model-based and model-free) · Offline RL, general function approximation | value suboptimality: both achieve $O(1/\sqrt{N})$ suboptimality | Helped |
| Add state-wise augmented Lagrangian occupancy-validity penalties | Offline contextual bandits with MIS · Offline contextual bandits | value suboptimality: $O(1/\sqrt{N})$ | Helped |
| Add an augmented Lagrangian penalty for occupancy normalization | Offline multi-armed bandits with MIS · Offline multi-armed bandits | value suboptimality: $O(1/\sqrt{N})$ | Helped |
| Remove behavior regularization without enforcing occupancy validity (ablation) | MIS · Offline multi-armed bandits | value suboptimality: ${1}/{6}$ | Hurt |
| Restrict the unregularized MIS search to weights inducing valid probability distributions (ablation) | Unregularized MIS · Offline multi-armed bandits | value suboptimality: $O(1/\sqrt{N})$ | Helped |
| Use behavior regularization instead of enforcing state-wise validity with augmented Lagrangian (ablation) | MIS · Offline contextual bandits | value suboptimality: $\Omega(N^\beta)$ with $\beta > -1/2$ | Hurt |
| Add behavior regularization to MIS (ablation) | PRO-MAB · Offline multi-armed bandits | value suboptimality: $O(1/\sqrt{N})$ | Helped |
Closest papers by meaning
- Revisiting the Linear-Programming Framework for Offline RL with General Function Approximation (ICML 2023 · similarity 0.94)
- Offline RL Policies Should Be Trained to be Adaptive (ICML 2022 · similarity 0.93)
- On Sample-Efficient Offline Reinforcement Learning: Data Diversity, Posterior Sampling and Beyond (NeurIPS 2023 · similarity 0.93)
- Minimax Optimal and Computationally Efficient Algorithms for Distributionally Robust Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.92)
- On the Complexity of Offline Reinforcement Learning with Q*-Approximation and Partial Coverage (NeurIPS 2026 · similarity 0.92)
- Compositional Conservatism: A Transductive Approach in Offline Reinforcement Learning (ICLR 2024 · similarity 0.92)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.