Research map / Reinforcement learning / Offline RL
Behavior Estimation from Multi-Source Data for Offline Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace one global behavior estimate with trajectory-assigned behavior policies | BRAC-v (LBRAC-v) · D4RL datasets | normalized return | Helped |
| use a single behavior policy rather than a behavior set | offline RL on heterogeneous-k data · heterogeneous-k | relative return: When k=5, BRAC-v lost about 50% of the performance when k=1. | Hurt |
| use behavior-set estimation instead of VAE-based behavior modeling | offline RL · medium and medium-replay D4RL datasets | normalized return | Helped |
| use VAE-based modeling rather than the behavior set | offline RL · large, high-quality medium-expert datasets | normalized return | Helped |
| increase the number of behavior embeddings K (ablation) | LBRAC-v · D4RL datasets | normalized return: K=1, K=5, K=10, K=15, K=20 values are reported in the varying-K table. | Mixed |
| increase K, encouraging policy-based rather than reward-based clustering (ablation) | proposed behavior model · walker2d medium-expert | reward-related pattern visibility | Hurt |
| share a trajectory embedding across actions in the trajectory | LBRAC-v · walker2d D4RL | reward-related trajectory clustering | Helped |
| add policy-learning information to the proposed model | LBRAC-v · walker2d medium-expert | reward-related pattern visibility | Helped |
Closest papers by meaning
- Efficient Policy Evaluation with Offline Data Informed Behavior Policy Design (ICML 2024 · similarity 0.91)
- Leveraging Offline Data in Online Reinforcement Learning (ICML 2023 · similarity 0.91)
- Hybrid Reinforcement Learning from Offline Observation Alone (ICML 2024 · similarity 0.90)
- Iteratively Refined Behavior Regularization for Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.90)
- Simple Ingredients for Offline Reinforcement Learning (ICML 2024 · similarity 0.90)
- Constrained Latent Action Policies for Model-Based Offline Reinforcement Learning (NeurIPS 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.