Research map / Reinforcement learning / Visual and representation RL
CrossLoco: Human Motion Driven Control of Legged Robots via Guided Unsupervised Reinforcement Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CrossLoco, a guided unsupervised RL framework that learns a quadruped controller driven by human motion by maximizing mutual information via cycle-consistency between robot and human poses.
Problem
Cross-morphology human motion driven control (HMDC) of legged robots requires establishing human-robot correspondence without a predefined robot motion dataset or paired data.
Approach
A cycle-consistency correspondence reward built from two learned mappers (robot-to-human and human-to-robot) that is added to PPO-based skill-conditioned RL, so the policy learns robot skills while simultaneously discovering correspondence to human motions, with a root-tracking reward to preserve high-level semantics.
Key results
- CrossLoco achieves ACR 0.785 vs 0.558 for DeepMimic and DIV 2.853 vs 2.231, with the best root tracking reward (0.743).
- In a 15-subject user study, CrossLoco was preferred 43% of the time vs 16% for DeepMimic, though DeepMimic had the highest correct rate (67.5% vs 65.5%).
- Trained on a single RTX 3080 Ti with 1024 parallel environments for about 3 days.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the correspondence reward (ablation) | CrossLoco · LaFAN1 to Aliengo | ACR: 0.556 -> 0.785 | Helped |
| add the correspondence reward (ablation) | CrossLoco · LaFAN1 to Aliengo | DIV: 2.494 -> 2.853 | Helped |
| add the correspondence reward (ablation) | CrossLoco · LaFAN1 to Aliengo | CR: 30.0\% -> 65.5\% | Helped |
| add the correspondence reward (ablation) | CrossLoco · LaFAN1 to Aliengo | PR: 14\% -> 43\% | Helped |
| add the correspondence reward (ablation) | CrossLoco · LaFAN1 to Aliengo | RTR: 0.740 -> 0.743 | No clear effect |
| add robot-pose cycle consistency (ablation) | CrossLoco · LaFAN1 to Aliengo | ACR: 0.683 -> 0.785 | Helped |
| add robot-pose cycle consistency (ablation) | CrossLoco · LaFAN1 to Aliengo | DIV: 2.779 -> 2.853 | Helped |
| add robot-pose cycle consistency (ablation) | CrossLoco · LaFAN1 to Aliengo | PR: 27\% -> 43\% | Helped |
| replace manually engineered retargeting plus DeepMimic with CrossLoco | Aliengo · LaFAN1 to Aliengo | ACR: 0.558 -> 0.785 | Helped |
| replace manually engineered retargeting plus DeepMimic with CrossLoco | Aliengo · LaFAN1 to Aliengo | CR: 67.5\% -> 65.5\% | Hurt |
Surprising findings
- DeepMimic, despite engineered retargeting, had the highest correct-match rate in the user study while being least preferred (16%).
Limitations
- Struggles to transfer high-frequency movements such as hopping.
- Cannot follow sharp backward turns and 180-degree orientation changes.
- Evaluated only in simulation (Isaac Gym); real-world deployment left to future work.
Experimental setup
Datasets: LaFAN1
Metrics: Averaged Correspondence Reward (ACR) Diversity (DIV) Averaged Root Tracking Reward (RTR) Correct Rate (CR) Preference (PR)
Baselines: DeepMimic Task-Only R2H-Only
Builds on: PPO DeepMimic cycle-consistency variational mutual information skill discovery
Closest papers by meaning
- PEAC: Unsupervised Pre-training for Cross-Embodiment Reinforcement Learning (NeurIPS 2024 · similarity 0.88)
- SUGAR: A Scalable Human-Video-Driven Generalizable Humanoid Loco-Manipulation Learning Framework (NeurIPS 2026 · similarity 0.86)
- Adversarial Locomotion and Motion Imitation for Humanoid Policy Learning (NeurIPS 2025 · similarity 0.86)
- Universal Humanoid Motion Representations for Physics-Based Control (ICLR 2024 · similarity 0.86)
- DUO: Diverse, Uncertain, On-Policy Query Generation and Selection for Reinforcement Learning from Human Feedback (AAAI 2025 · similarity 0.86)
- Foundation Policies with Hilbert Representations (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.