Research map / Reinforcement learning / Offline RL
From Static Constraints to Dynamic Adaptation: Sample-Level Constraint Relaxation for Offline-to-Online Reinforcement Learning
Lipeng Zu, YU QIAN, Shayok Chakraborty, Xiaonan Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add DARE sample-level exchange to Cal-QL | Cal-QL · D4RL total, 0.2M online steps | total normalized score: 1272.7 -> 1415.4 | Improved |
| Add DARE sample-level exchange to IQL | IQL · D4RL total, 0.2M online steps | total normalized score: 1085.3 -> 1253.0 | Improved |
| Add behavior-consistent exchange to DARE-I with SAC objective removed (ablation) | DARE-I w/o SAC · Adroit human-v0 total | total normalized score: 106.67 -> 113.90 | Improved |
| Add exchange to DARE-I with PEX removed (ablation) | DARE-I w/o PEX · Adroit human-v0 total | total normalized score: 8.96 -> 56.02 | Improved |
| Add exchange to TD-only DARE-I (no SAC, no PEX) (ablation) | DARE-I w/o S&P · Adroit human-v0 total | total normalized score: 104.25 -> 122.15 | Improved |
| Use posterior-induced exchange instead of monotonic Gaussian or direct distance exchange (ablation) | DARE-C · D4RL locomotion (replay and medium datasets) | normalized score stability | Improved |
| Use behavior-consistent exchange rather than reversed (curiosity-style) assignment (ablation) | DARE-I · D4RL locomotion total, diffusion behavior model | total normalized score: 791.1 -> 819.8 | Improved |
| Use behavior-consistent exchange rather than reversed assignment, online actor as behavior model (ablation) | DARE-I · D4RL locomotion total, policy behavior model | total normalized score: 766.5 -> 812.1 | Improved |
| Add exchange mechanism to DARE-C and DARE-I (ablation) | DARE-C, DARE-I · D4RL locomotion replay and medium datasets | average score difference | Improved |
| Use online policy actor instead of offline diffusion model as behavior model (ablation) | DARE · D4RL locomotion | total normalized score: 819.8 -> 812.1 | Hurt |
| Use DARE-I in place of PEX on hopper-medium-expert | IQL group · hopper-medium-expert | normalized score: 90.2 -> 104.5 | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- Cal-QL — Cal-QL: Calibrated Offline RL Pre-Training for Efficient Online Fine-Tuning (NeurIPS 2023) · better on 2 of 2 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- A Clean Slate for Offline Reinforcement Learning (NeurIPS 2025 · similarity 0.90)
- Iteratively Refined Behavior Regularization for Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.89)
- Localized Dynamics-Aware Domain Adaption for Off-Dynamics Offline Reinforcement Learning (NeurIPS 2026 · similarity 0.89)
- From Static Policies to Adaptive Priors in Offline Reinforcement Learning (NeurIPS 2026 · similarity 0.89)
- Information-Directed Offline-to-Online Reinforcement Learning (NeurIPS 2026 · similarity 0.89)
- Simple Ingredients for Offline Reinforcement Learning (ICML 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
