Research map / Reinforcement learning / Visual and representation RL
Videos are Sample-Efficient Supervisions: Behavior Cloning from Videos via Latent Representations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BCV-LR, an unsupervised sample-efficient imitation-from-videos framework that pre-trains self-supervised encoders and latent actions from action-free videos, then aligns latent actions to real actions online via behavior cloning with 100k environment steps.
Problem
Imitation learning from action-free and reward-free videos with very limited environment interactions, where visual complexity and missing action labels make policy learning sample-inefficient.
Approach
Learns latent actions between consecutive video frames via a VQ-discretized latent action predictor and a world model trained with a dynamics objective, then online fine-tunes these latent actions with reward-free interactions, decodes them to real actions, and iteratively improves a latent behavior-cloned policy.
Key results
- On 16 Procgen tasks with 100k interactions, BCV-LR averages 13.8 return (0.79 of expert video performance), exceeding UPESV (9.0) and PPO (2.3).
- On 8 DMControl tasks at 100k steps, BCV-LR averages 604 vs 310 for TACO and 158 for LAIFO, with the best result on 6/8 tasks versus RL baselines at 100 steps.
- On 4 Metaworld tasks at 50k steps, BCV-LR achieves a mean success rate of 0.84 versus 0.16 for DrQv2 and 0.07 for BCO.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use BCV-LR rather than UPESV | Procgen policy learning · Procgen, 100k steps (Mean) | score: 9.0 -> 13.8 | Helped |
| use BCV-LR rather than BCO | DMControl policy learning · DMControl, 100k steps (Mean) | score: 336 -> 604 | Helped |
| remove self-supervised visual pre-training (ablation) | BCV-LR · Procgen and continuous control | policy performance | Hurt |
| remove latent action pre-training (ablation) | BCV-LR · Procgen and continuous control | sample efficiency | Hurt |
| fix pre-trained latent actions and only update the decoder (ablation) | BCV-LR · Procgen and continuous control | policy performance | Hurt |
| add self-reconstruction loss to contrastive feature pre-training (ablation) | BCV-LR · Procgen video games | policy performance | Helped |
| constrain online latent-action fine-tuning with the pre-trained world model (ablation) | BCV-LR · policy learning | policy performance | Helped |
| fine-tune rather than freeze self-supervised features (ablation) | BCV-LR · reacher_hard and finger_spin | score: reacher_hard: 876 ± 15; 906 ± 65; 900 ± 31; finger_spin: 937 ± 26; 920 ± 57; 942 ± 48 | No clear effect |
| reduce interactions per online cycle from 1000 to 2 (ablation) | BCV-LR · reacher_hard and finger_spin | score: reacher_hard: 900 ± 31 -> 875 ± 65; finger_spin: 942 ± 48 -> 956 ± 20 | Mixed |
| use 10k offline action-labeled expert transitions for fine-tuning and cloning | BCV-LR · reacher_hard | score: 900 ± 31 -> 938 ± 44 | Helped |
| use multi-task instead of single-task pre-training | BCV-LR · Bigfish, Maze, Starpilot, Bossfight, and Dodgeball | score: Bigfish: 35.9 ± 2.0 -> 32.2 ± 2.0; Maze: 9.9 ± 0.1 -> 9.6 ± 0.1; Starpilot: 54.8 ± 1.4 -> 44.3 ± 1.9; Bossfight: 10.3 ± | Hurt |
| add offline action-labeled expert transitions | BCV-LR · finger_spin | score: 942 ± 48 -> 978 ± 7 | Helped |
Surprising findings
- Behavior cloning from videos with BCV-LR outperforms RL methods given ground-truth rewards in the low-interaction regime.
- Fine-tuning self-supervised encoder features yields no clear gain on policy performance, and freezing them is comparable.
Limitations
- Suffers from covariate shift inherent to behavior cloning, limiting performance on long-horizon sequential tasks like walker_walk at 500k steps.
- Performance bottleneck when video data is reduced to 5k transitions.
- Code and data not yet released (promised after acceptance).
Experimental setup
Datasets: Procgen expert videos (8M steps) DMControl expert videos (200k frames) Metaworld expert videos
Benchmarks: Procgen DMControl Metaworld
Metrics: episode return success rate video-normalized mean score
Baselines: UPESV BCO ILPO LAIFO LAPO PPO DrQv2 TACO
Builds on: UPESV LAPO TACO DrQv2 Sinkhorn-Knopp
Closest papers by meaning
- LAFP: Preserving Latent Action Structure in Latent Policy Learning via Flow Matching (NeurIPS 2026 · similarity 0.91)
- Rethinking Latent Redundancy in Behavior Cloning: An Information Bottleneck Approach for Robot Manipulation (ICML 2025 · similarity 0.90)
- Learning an Actionable Discrete Diffusion Policy via Large-Scale Actionless Video Pre-Training (NeurIPS 2024 · similarity 0.90)
- Is Behavior Cloning All You Need? Understanding Horizon in Imitation Learning (NeurIPS 2024 · similarity 0.89)
- Latent Action Learning Requires Supervision in the Presence of Distractors (ICML 2025 · similarity 0.89)
- Interactive and Hybrid Imitation Learning: Provably Beating Behavior Cloning (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.