Research map / Reinforcement learning / Imitation learning
Split the Differences, Pool the Rest: Provably Efficient Multi-Objective Imitation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Multi-Output Augmented Behavioral Cloning (MA-BC), which splits divergent state-action data and pools conflict-free data across experts, proving Pareto-optimal recovery at faster rates and a matching minimax lower bound for offline multi-objective imitation.
Problem
Recovering policies on the Pareto front from demonstrations of multiple Pareto-optimal experts in MOMDPs; naive pooling yields dominated policies and independent per-expert BC is sample-inefficient.
Approach
Identify states where experts disagree (diverging dataset), pool all other state-action pairs as a common dataset shared across experts, and run BC per expert on common data plus its own expert data; proves a Pareto Path structural result and a matching minimax lower bound.
Key results
- Naive BC fails to converge to the epsilon-Pareto front as data grows, while MA-BC converges at a faster statistical rate than Isolated BC across discrete environments.
- Performance gap between MA-BC and Isolated BC widens as the concentrability coefficient decreases (more overlap between experts' occupancy measures).
- On the continuous LQR drone task, a moderate spatial tolerance delta improves recovery of the Economic expert in low-data regimes, with optimal delta shrinking as data grows.
Surprising findings
- Naively pooling conflicting expert data (single-output BC) can yield policies off the Pareto front even with infinite data.
- Neighboring Pareto-optimal policies can differ in more than one state in general, so the single-state-difference property of prior work does not hold without extra assumptions.
Limitations
- Continuous extension is a heuristic without formal guarantees.
- Theory is for discrete MOMDPs with deterministic experts in the main text.
- Experiments use synthetic experts and limited environments; MA-BC is not always better than reducing to single-expert IL when concentrability is high.
Experimental setup
Datasets: Deep Sea Treasure Resource Gathering Slippery Y-Maze LQR 6-DOF quadcopter
Benchmarks: Deep Sea Treasure Resource Gathering Slippery Y-Maze LQR drone control
Metrics: L-infinity distance to Pareto front (normalized returns)
Baselines: Naive BC Isolated BC
Builds on: behavioral cloning Pareto front traversal minimax lower bounds for imitation learning (Rajaraman et al.)
Closest papers by meaning
- Behavioral Cloning from Noisy Demonstrations (ICLR 2021 · similarity 0.90)
- Interactive and Hybrid Imitation Learning: Provably Beating Behavior Cloning (NeurIPS 2025 · similarity 0.89)
- Causal Imitation for Markov Decision Processes: a Partial Identification Approach (NeurIPS 2024 · similarity 0.88)
- Is Behavior Cloning All You Need? Understanding Horizon in Imitation Learning (NeurIPS 2024 · similarity 0.88)
- Diverse Policies Recovering via Pointwise Mutual Information Weighted Imitation Learning (ICLR 2025 · similarity 0.88)
- Diffusion Model-Augmented Behavioral Cloning (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.