Research map / Reinforcement learning / Imitation learning
Zero-Shot Offline Imitation Learning via Optimal Transport
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ZILOT, which performs zero-shot offline imitation from a single action-free, rough, partial expert demonstration by planning with a learned world model and minimizing an optimal-transport distance between the policy's and expert's occupancies, avoiding the myopia of hierarchical goal-reaching decompositions.
Problem
Zero-shot offline imitation from a single rough, partial, action-free expert demonstration; existing hierarchical goal-sequence methods behave myopically when goals are abstracted, failing to achieve later goals.
Approach
Lift a learned goal-conditioned value function to a state-goal distance and use it as the cost in an entropic optimal transport problem between the expert goal sequence and the occupancy of rollouts from a learned world model, optimized by zero-order MPC (iCEM) at inference time.
Key results
- ZILOT achieves lowest W_min on the aggregate fetch_slide_large_2D tasks (0.098 vs 0.205 for MPC+Cls and 0.238 for Pi+Cls) and a GoalFraction of 0.70 vs 0.45 for MPC+Cls.
- On halfcheetah, ZILOT's aggregate GoalFraction is 0.82 vs 0.71 for MPC+Cls and 0.70 for Pi+Cls.
- Proposition shows a hierarchical goal-reaching policy with abstraction can fail to reach a goal for all rollout lengths, motivating the occupancy-matching formulation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace learned distance d with goal metric h (ablation) | ZILOT · fetch_pick_and_place-all | W_min: 0.070±0.009 -> 0.207±0.026 | Hurt |
| use a goal classifier to discard reached goals (ablation) | ZILOT · fetch_pick_and_place-all | W_min: 0.070±0.009 -> 0.115±0.007 | Hurt |
| use unbalanced optimal transport (ablation) | ZILOT · fetch_pick_and_place-all | W_min: 0.070±0.009 -> 0.068±0.008 | No clear effect |
| replace learned distance d with goal metric h (ablation) | ZILOT · fetch_pick_and_place-all | GoalFraction: 0.72±0.04 -> 0.32±0.06 | Hurt |
| use a goal classifier to discard reached goals (ablation) | ZILOT · fetch_pick_and_place-all | GoalFraction: 0.72±0.04 -> 0.56±0.02 | Hurt |
| use unbalanced optimal transport (ablation) | ZILOT · fetch_pick_and_place-all | GoalFraction: 0.72±0.04 -> 0.73±0.05 | No clear effect |
| use occupancy matching instead of hierarchical goal-reaching planning | zero-shot imitation planners · fetch_slide_large_2D-all | W_min: 0.205±0.020 -> 0.098±0.007 | Helped |
| use occupancy matching instead of hierarchical goal-reaching planning | zero-shot imitation planners · fetch_slide_large_2D-all | GoalFraction: 0.45±0.04 -> 0.70±0.02 | Helped |
Surprising findings
- Reward-based FB imitation baselines (ER_FB, RER_FB) perform substantially worse than myopic goal-based planners, attributed to regularized f-divergence yielding degenerate solutions in ergodic MDPs.
- Using the goal metric h as OT cost underperforms the learned value-based distance d, attributed to flat optimization landscapes in goal space.
Limitations
- Relies on a learned world model whose accuracy degrades with horizon, forcing a finite planning horizon H that can reintroduce myopia.
- Induces non-Markovian policies, which are a larger search space than necessary.
- Performance plateaus beyond horizon 16 in fetch_slide_large_2D, suggesting model inaccuracy limits planning.
Experimental setup
Datasets: offline suboptimal trajectories from fetch, halfcheetah, pointmaze environments
Benchmarks: fetch pick-and-place fetch push fetch slide large 2D halfcheetah pointmaze medium walker
Metrics: W_min (Wasserstein-1 distance) GoalFraction
Baselines: Policy+Cls MPC+Cls ER_FB RER_FB
Builds on: TD-MPC2 forward-backward representations iCEM Sinkhorn algorithm
Closest papers by meaning
- Consistent Zero-Shot Imitation with Contrastive Goal Inference (ICML 2026 · similarity 0.92)
- GO-DICE: Goal-Conditioned Option-Aware Offline Imitation Learning via Stationary Distribution Correction Estimation (AAAI 2024 · similarity 0.90)
- SEABO: A Simple Search-Based Method for Offline Imitation Learning (ICLR 2024 · similarity 0.89)
- Is Behavior Cloning All You Need? Understanding Horizon in Imitation Learning (NeurIPS 2024 · similarity 0.89)
- OLLIE: Imitation Learning from Offline Pretraining to Online Finetuning (ICML 2024 · similarity 0.88)
- Interactive and Hybrid Imitation Learning: Provably Beating Behavior Cloning (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.