Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Deep Demonstration Tracing: Learning Generalizable Imitator Policy for Runtime Imitation from a Single Demonstration
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Deep Demonstration Tracing (DDT), a demonstration-transformer policy trained with SAC under a context-based meta-RL and OSIL-reward setup, for one-shot imitation that generalizes to unforeseen changes after the demonstration.
Problem
One-shot imitation learning often assumes demonstration-time and deployment-time situations are similar; it fails when unforeseen changes (e.g., obstacles) occur after the demonstration, requiring the imitator to generalize beyond demonstrated states.
Approach
A demonstration transformer whose attention structure mirrors a 3-stage human-like tracing process (identify relevant expert states, analyze expert behavior, trace back to the demonstration), combined with training the imitator as a context-based meta-RL policy with SAC and a stationary imitation reward, rather than behavior cloning.
Key results
- On Meta-World with disturbance, DDT reaches 0.61±0.20 success on unseen demos versus 0.12 for DCRL, 0.10 for CbMRL and 0.00 for Trans4OSIL.
- On VPAM training tasks, performance degradation from training to unforeseen obstacles is -15% for DDT versus -20%, -33%, and -52% for the baselines.
- Scaling experiments show roughly log-linear performance gains with more demonstrations or model parameters on VPAM.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use DDT instead of the strongest baseline | VPAM imitation policy · VPAM Train | success rate: 0.84 -> 0.86 | Helped |
| use DDT instead of the strongest baseline | VPAM imitation policy · VPAM Non-Obstacle | success rate: 0.71 -> 0.73 | Helped |
| use DDT instead of the strongest baseline | VPAM imitation policy · VPAM Unforeseen Obstacle | success rate: 0.57 -> 0.62 | Helped |
| use DDT instead of DCRL | Meta-World imitation policy · Meta-World, with disturbance, unseen demonstrations | success rate: 0.12±0.14 -> 0.61±0.20 | Helped |
| replace the demonstration transformer with a standard Transformer (ablation) | DDT · VPAM, multi-map without obstacles, unseen maps | success rate | Hurt |
| remove the OSIL reward and train with only the ending reward (ablation) | DDT · VPAM, multi-map without obstacles, unseen maps | learning efficiency | Hurt |
| increase the number of demonstrations | DDT · VPAM, multi-map with obstacles, unseen maps | success rate | Helped |
| increase the model parameters | DDT · VPAM, multi-map with obstacles, unseen maps | success rate: approximately 2× performance improvement | Helped |
Surprising findings
- Behavior cloning (Trans4OSIL) performed far worse than RL-based baselines on the navigation tasks, attributed to dead states where small imitation errors cause failure.
Limitations
- Performance drops in partially observable settings (0.67 success on partially observable VPAM).
- Assumes expert demonstrations and a recoverable MDP set for the theoretical guarantee.
- Visual OSIL not addressed; future work suggested for partial observability.
Experimental setup
Datasets: Valet Parking Assist in Maze (VPAM) demonstrations Meta-World demonstrations Gymnasium Reacher and Pusher Complex planning robot manipulation task (Pang et al., 2023)
Benchmarks: Valet Parking Assist in Maze (VPAM) Meta-World Gymnasium Reacher and Pusher complex planning robot manipulation partially observable VPAM
Metrics: success rate performance degradation under unforeseen obstacles
Baselines: DCRL Trans4OSIL CbMRL Nearest-neighbor replay (Johns 2021; Valassakis 2022)
Builds on: Soft Actor-Critic context-based meta-RL Transformer one-shot imitation learning
Closest papers by meaning
- OSVI-WM: One-Shot Visual Imitation for Unseen Tasks using World-Model-Guided Trajectory Generation (NeurIPS 2025 · similarity 0.90)
- Instant Policy: In-Context Imitation Learning via Graph Diffusion (ICLR 2025 · similarity 0.88)
- Agnostic Interactive Imitation Learning: New Theory and Practical Algorithms (ICML 2024 · similarity 0.88)
- Expert Proximity as Surrogate Rewards for Single Demonstration Imitation Learning (ICML 2024 · similarity 0.88)
- Policy Decorator: Model-Agnostic Online Refinement for Large Policy Model (ICLR 2025 · similarity 0.87)
- Learning from Demonstrations via Capability-Aware Goal Sampling (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.