Research map / Reinforcement learning / Imitation learning
Reinforcement Learning from Imperfect Demonstrations under Soft Expert Guidance
Mingxuan Jing, Xiaojian Ma, Wenbing Huang, Fuchun Sun, Chao Yang, Bin Fang, Huaping Liu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use expert guidance as soft constraint instead of penalty (reward-shaping) term | RLfD with imperfect demonstrations · Physical control benchmarks (Hopper, Walker2d, Ant) | exact reward: e.g. 2645.58 vs -68.59 on the 111-dim task | Improved |
| Soft constraint vs Pre-train baseline (continued guidance during training) | RLfD · Hopper, Walker2d | reward improvement over training: Hopper +973.42, Walker2d +876.40 | Improved |
| Anneal constraint tolerance instead of fixing it (epsilon>0 vs 0) (ablation) | Soft constraint RLfD · HalfCheetah | reward | Improved |
| Use too large initial tolerance d0 (e.g. 1) (ablation) | Soft constraint RLfD · HalfCheetah | reward | Hurt |
| Use too small tolerance d0 with imperfect demonstrations (ablation) | Soft constraint RLfD · HalfCheetah | final reward | Hurt |
| Vary annealing factor epsilon around good values (ablation) | Soft constraint RLfD · HalfCheetah | reward | No clear effect |
| Increase number of demonstration state-action pairs (50 to 5000) (ablation) | Soft constraint RLfD vs baselines · HalfCheetah | reward | Improved |
| Raise demonstration quality by mixing in more expert data (ablation) | All RLfD methods · HalfCheetah | reward | Improved |
Closest papers by meaning
- RLfOLD: Reinforcement Learning from Online Demonstrations in Urban Autonomous Driving (AAAI 2024 · similarity 0.89)
- Is Inverse Reinforcement Learning Harder than Standard Reinforcement Learning? A Theoretical Perspective (ICML 2024 · similarity 0.89)
- Learning from Ambiguous Demonstrations with Self-Explanation Guided Reinforcement Learning (AAAI 2024 · similarity 0.89)
- Hybrid Inverse Reinforcement Learning (ICML 2024 · similarity 0.89)
- Demonstration-Regularized RL (ICLR 2024 · similarity 0.88)
- Recent Advancements in Inverse Reinforcement Learning (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
