Research map / Reinforcement learning / Offline RL
The Provable Benefit of Unsupervised Data Sharing for Offline Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Pessimistic reward penalty (PDS) vs naive reward prediction | IQL on Meta-World multi-task · Meta-World (4 tasks) | average normalized score: +13.6 over Rew Pred (119.1 vs 105.5) | Helped |
| Zero-reward data sharing (UDS) vs no sharing | IQL on Walker2d/Hopper mixed-quality data · Walker2d expert/random | normalized score | Hurt |
| Pessimistic reward penalty (PDS) vs UDS | IQL on AntMaze medium-diverse directed · AntMaze medium-diverse directed | normalized score: +19.9 over CDS+UDS (53.2 vs 33.3) | Helped |
| Pessimistic reward penalty (PDS) vs CDS+UDS | IQL on AntMaze medium-play undirected · AntMaze medium-play undirected | normalized score: -8.0 vs CDS+UDS (29.2 vs 37.2) | Hurt |
| Pessimistic reward penalty (PDS) vs naive reward prediction | IQL on Hopper medium-labeled with medium unlabeled (0.4M) · Hopper medium/50K + medium/0.4M | normalized score: +8.9 over Rew Pred (77.8 vs 68.9) | Helped |
| Pessimistic reward penalty (PDS) vs naive reward prediction on Walker2d random-labeled/expert-unlabeled | IQL · Walker2d random/50K + expert/0.1M | normalized score: +6.1 over Rew Pred (101.4 vs 95.3) | Helped |
| Pessimistic reward penalty (PDS) vs naive reward prediction on Hopper expert-labeled/random-unlabeled | IQL · Hopper expert/50K + random/0.1M | normalized score: -27.9 vs UDS-style (42.7 vs 53.3 UDS, PDS below best); above Rew Pred 27.9 by +14.8 | Hurt |
| Pessimistic reward penalty (PDS) on Walker2d expert-labeled/random-unlabeled | IQL · Walker2d expert/50K + random/0.1M | normalized score: +13.8 over UDS (39.5 vs 25.7) | Helped |
| Penalty weight k sweep (interpolating UDS and reward prediction) (ablation) | PDS on Hopper/Walker2d · Hopper/Walker2d labeled/unlabeled mixes | normalized score | Helped |
| Adaptive k (based on mean reward difference) vs fixed k | PDS · Hopper/Walker2d | normalized score | No clear effect |
Closest papers by meaning
- Conservative Data Sharing for Multi-Task Offline Reinforcement Learning (NeurIPS 2021 · similarity 0.92)
- How to Leverage Unlabeled Data in Offline Reinforcement Learning (ICML 2022 · similarity 0.92)
- A Clean Slate for Offline Reinforcement Learning (NeurIPS 2025 · similarity 0.90)
- Is Value Learning Really the Main Bottleneck in Offline RL? (NeurIPS 2024 · similarity 0.90)
- Hybrid Reinforcement Learning from Offline Observation Alone (ICML 2024 · similarity 0.89)
- Efficient Online Reinforcement Learning Fine-Tuning Need Not Retain Offline Data (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.