Research map / 3D vision / Human pose estimation
Imitation Learning from a Single Temporally Misaligned Video
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ORCA, a dense per-timestep reward from a single misaligned video that scores the probability the learner covers demonstration subgoals in order, used to train RL agents on Meta-world and Humanoid tasks.
Problem
Learning sequential tasks from a single visual demonstration that is temporally misaligned (timing, embodiment, execution differences), where frame-level distribution matching (OT, DTW, TemporalOT) fails to enforce subgoal ordering and full coverage.
Approach
Define imitation reward at the sequence level: a recursive dynamic-programming ordered-coverage probability that the learner has covered all demonstration frames in the correct order, with a final-subgoal-occupancy term; pretrain on TemporalOT then fine-tune with ORCA reward.
Key results
- On misaligned Meta-world demos, ORCA reaches 0.50 average normalized return vs 0.11 for TemporalOT (best frame-level baseline), a 4.5x improvement.
- On Humanoid-v4 tasks, ORCA (NP) achieves 62.9 average return and ORCA 43.3, versus 6.55 for TemporalOT (about 6.6x for ORCA).
- On temporally aligned demos, ORCA averages 0.57 normalized return compared with 0.33 for ORCA without pretraining.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace TemporalOT frame-level matching with ORCA ordered-coverage rewards | Meta-world policies · Meta-world temporally misaligned demonstrations | expert-normalized return: 0.11 -> 0.50 | Helped |
| replace TemporalOT frame-level matching with ORCA ordered-coverage rewards | Humanoid-v4 policies · Humanoid-v4 temporally misaligned demonstrations | return: 6.55 -> 43.3 | Helped |
| add TemporalOT pretraining before ORCA reward training (ablation) | ORCA · Meta-world temporally misaligned demonstrations | expert-normalized return: 0.49 (0.04) -> 0.50 (0.04) | No clear effect |
| add TemporalOT pretraining before ORCA reward training (ablation) | ORCA · Humanoid-v4 temporally misaligned demonstrations | return: 62.9 (7.02) -> 43.3 (4.61) | Hurt |
| add TemporalOT pretraining before ORCA reward training (ablation) | ORCA · Aligned demonstrations | expert-normalized return: 0.33 -> 0.57 | Helped |
| use four demonstrations with different speeds instead of one demonstration (ablation) | ORCA · Meta-world, three tasks | expert-normalized return: 0.67 ± 0.08 -> 0.77 ± 0.08 | Helped |
| use four demonstrations with different speeds instead of one demonstration (ablation) | TemporalOT · Meta-world, three tasks | expert-normalized return: 0.14 ± 0.02 -> 0.16 ± 0.02 | No clear effect |
| condition the ORCA policy on image observations instead of state observations (ablation) | ORCA · Meta-world | expert-normalized return: 0.704 ± 0.10 -> 0.76 ± 0.06 | No clear effect |
| use ORCA rather than TemporalOT as misalignment increases | Meta-world policies · Door-open, Window-open, and Lever-pull under varying temporal misalignment | expert-normalized return | Helped |
| use ResNet50 rather than LIV or DINOv2 as the visual encoder (ablation) | ORCA · Visual encoder ablation | return | Mixed |
Surprising findings
- TemporalOT often fails under misalignment because it forces equal time per subgoal, causing agents to run out of timesteps.
- On Humanoid, pretraining with TemporalOT hurts, so ORCA trained from scratch (ORCA NP) performs better.
Limitations
- Relies on the quality of the visual distance metric; noisy encoders degrade rewards.
- Video-following may fail due to task misspecification; no language subgoals.
- Performance degrades for slower demonstrations with pauses; agents capture general motion but miss fine details.
Experimental setup
Datasets: Meta-world demonstrations generated by hand-engineered policies Humanoid-v4 interpolated demonstrations Humanoid joint predictor dataset (9,038 samples)
Benchmarks: Meta-world (10 tasks) Humanoid-v4 (4 tasks)
Metrics: expert-normalized return unnormalized return success rate
Baselines: OT TemporalOT DTW RoboCLIP Threshold reward ORCA (NP)
Builds on: optimal transport dynamic time warping TemporalOT DrQ-v2 SAC
Closest papers by meaning
- Cross-domain Imitation from Observations (ICML 2021 · similarity 0.89)
- Videos are Sample-Efficient Supervisions: Behavior Cloning from Videos via Latent Representations (NeurIPS 2025 · similarity 0.89)
- Of Moments and Matching: A Game-Theoretic Framework for Closing the Imitation Gap (ICML 2021 · similarity 0.88)
- Imitation Learning from Observation with Automatic Discount Scheduling (ICLR 2024 · similarity 0.88)
- Efficient Imitation under Misspecification (ICLR 2025 · similarity 0.87)
- OSVI-WM: One-Shot Visual Imitation for Unseen Tasks using World-Model-Guided Trajectory Generation (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.