Research map / Reinforcement learning / Imitation learning
Reconciling Safety and Performance via Dual-Expert Offline Imitation Learning
Seokin Seo, Sung Kuk Shyn, Minji Seo, Yisoo Lee, Jongmin Lee, Hongseok Yang, Kee-Eung Kim
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a dual-expert offline imitation learning method that reconciles safety and performance without explicit cost functions or jointly safe-and-high-performing expert demonstrations.
Problem
Offline imitation learning must achieve high returns while providing safety guarantees, but explicit cost functions are hard to design for complex safety constraints and demonstrations that are both safe and high-performing are costly to collect.
Approach
Uses two separate expert demonstration sources (dual experts) so that safety and performance can be learned from distinct data rather than requiring a single expert that is both safe and high-performing; avoids hand-designed cost functions.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace distribution matching to d_S with safe-support constraint (DexDICE vs DemoDICE-S) (ablation) | DexDICE · Re-curated DSRL, Safety Gymnasium | Avg return: 0.27 -> 0.37 | Improved |
| replace distribution matching to d_S with safe-support constraint (DexDICE vs DemoDICE-U) (ablation) | DexDICE · Re-curated DSRL, Safety Gymnasium | # Safe tasks: 10/21 -> 17/21 | Improved |
| use dual-expert DICE instead of pseudo-labelled safe RL (CDT/TraC) | DexDICE · Re-curated DSRL, Safety Gymnasium | Avg return: 0.19 -> 0.37 | Improved |
| use dual-expert DICE instead of pseudo-labelled safe RL (TraC) | DexDICE · Re-curated DSRL, MetaDrive | Avg return: 0.23 -> 0.50 | Improved |
| add support-filtering to BC (FBC) vs plain BC (ablation) | BC · Re-curated DSRL, Safety Gymnasium | Avg return: 0.46 -> 0.13 | Hurt |
| DexDICE vs DemoDICE-S on MetaDrive | DexDICE · Re-curated DSRL, MetaDrive | # Best among safe: 5/9 vs 5/9 | No clear effect |
| DexDICE on MetaDrive vs CDT/TraC cost | DexDICE · Re-curated DSRL, MetaDrive | # Safe tasks: 9/9 (CDT, TraC) vs 7/9 | Hurt |
| reduce safe demonstrations to N_S=2 (ablation) | DexDICE · Controlled Safety Gymnasium (6 tasks) | safety / return | Hurt |
| degrade quality of D_S or D_P demonstrations (ablation) | DexDICE · Controlled Safety Gymnasium (6 tasks) | safety and return | No clear effect |
| real-robot teleoperation vs baselines | DexDICE · Real mobile-robot navigation (10 trials) | goal reaching and violations | Improved |
Closest papers by meaning
- GO-DICE: Goal-Conditioned Option-Aware Offline Imitation Learning via Stationary Distribution Correction Estimation (AAAI 2024 · similarity 0.90)
- Interactive and Hybrid Imitation Learning: Provably Beating Behavior Cloning (NeurIPS 2025 · similarity 0.89)
- OLLIE: Imitation Learning from Offline Pretraining to Online Finetuning (ICML 2024 · similarity 0.88)
- Simplifying Constraint Inference with Inverse Reinforcement Learning (NeurIPS 2024 · similarity 0.88)
- How to Leverage Diverse Demonstrations in Offline Imitation Learning (ICML 2024 · similarity 0.88)
- RLIF: Interactive Imitation Learning as Reinforcement Learning (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
