Research map / Reinforcement learning / Imitation learning

Learning When to Stop: Selective Imitation Learning Under Arbitrary Dynamics Shift

NeurIPS 2026 · Reinforcement learning › Imitation learning · approach: Behavior cloning theory and models · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SeqRejectron, a sequential PQ-learning style selective imitation algorithm that abstains mid-trajectory via sparse validator policies, giving horizon-free guarantees under dynamics shift.

Problem

Imitation learning under arbitrary test-time dynamics shift, where the learner has labeled expert demonstrations only in the training environment and unlabeled state trajectories in the test environment; the learner must decide when to stop acting safely.

Approach

Certify trajectory prefixes with a sparse distribution over validator policies drawn from the training-consistent version space; the learner continues while all validators agree with the base policy and stops at the first disagreement. Extends Rejectron/PQ learning to sequential decisions with horizon-free rates, a cumulative Hellinger stopping time for stochastic policies, and regularized validat

Key results

• On LunarLander with theta=2, the source handoff rate drops near 0% while target handoff rises, letting the switched policy's cost move from the vanilla BC cost toward the expert's cost (20 trials, m=n=30).
• Deterministic theory: horizon-free sample complexity O~(sqrt(log|Pi|/n)) for stopping rate and regret when m ~ n, with H entering only via C_max.
• Stochastic lower bound: Omega(1/eps^3) labeled samples needed in the single-step stochastic setting, matching the cost-driven part of the upper bound.

Surprising findings

• Switched regret is driven to zero faster than stopped regret in the low-variance regime, which the authors find counterintuitive.

Limitations

• Stochastic setting lacks an oracle-efficient reduction; the cumulative Hellinger objective is computationally open.
• Gap between the variance-driven epsilon^-5 upper bound and the epsilon^-3 lower bound for stochastic policies.
• Experiments are a small proof-of-concept on LunarLander with a heuristic in place of the game-theoretic validator construction.

Experimental setup

Datasets: LunarLander-v3 simulated trajectories (source: windless; target: rightward wind)

Benchmarks: LunarLander-v3 with wind shift

Metrics: source handoff rate target handoff rate normalized switched cost mean handoff time

Baselines: vanilla behavior cloning per-step Rejectron reduction

Builds on: Rejectron PQ learning Hellinger-based imitation learning analysis DAgger

Closest papers by meaning

• Is Behavior Cloning All You Need? Understanding Horizon in Imitation Learning (NeurIPS 2024 · similarity 0.92)
• Behavior Cloning is Not All You Need: The Optimality of On-Policy Distillation for Noisy Expert Feedback (NeurIPS 2026 · similarity 0.91)
• Interactive and Hybrid Imitation Learning: Provably Beating Behavior Cloning (NeurIPS 2025 · similarity 0.90)
• Mitigating Covariate Shift in Behavioral Cloning via Robust Stationary Distribution Correction (NeurIPS 2024 · similarity 0.89)
• A Smooth Sea Never Made a Skilled SAILOR: Robust Imitation via Learning to Search (NeurIPS 2025 · similarity 0.89)
• Agnostic Interactive Imitation Learning: New Theory and Practical Algorithms (ICML 2024 · similarity 0.89)