Research map / Reinforcement learning / Imitation learning
Adversarial Locomotion and Motion Imitation for Humanoid Policy Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ALMI, an adversarial training framework where lower-body locomotion and upper-body motion-imitation policies are trained against each other with curricula, and releases the ALMI-X dataset (81K language-annotated whole-body trajectories) plus a preliminary Transformer foundation model.
Problem
Humanoid whole-body control that tracks human motions while keeping locomotion stable; joint whole-body RL policies are costly to train and prone to instability and falls on full-size robots.
Approach
Decouple the upper and lower body into two policies trained as a zero-sum Markov game: the locomotion policy treats upper-body motions as adversarial disturbances, and the upper-body policy treats velocity commands as adversarial disturbances, with iterative mutual updates and curriculum-based adversarial sampling in command/motion space instead of parameter space.
Key results
- On CMU MoCap in Isaac Gym (easy level), ALMI reaches linear velocity error 0.1135 and upper-body keypoint error 0.0460 versus Exbody's 0.2383 and 0.0995, with survival 1.0000 vs 0.8912.
- On Unitree G1 in simulation, ALMI achieves survival 0.9484 versus 0.8848 for Exbody2 and 0.3882 for OmniH2O.
- Iterative adversarial training improves lower-body policy: on hard commands, lower-3 achieves survival 0.9723 versus 0.8743 for lower-1 and 0.8480 without arm curriculum.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use iterative adversarial training through lower-3 + upper-2 rather than lower-2 + upper-2 (ablation) | ALMI · CMU, hard | E_vel (m/s): 0.2892 -> 0.2202 | Helped |
| Use iterative adversarial training through lower-3 + upper-2 rather than lower-1 + upper-2 (ablation) | ALMI · CMU, hard | E_vel (m/s): 0.2566 -> 0.2202 | Helped |
| Add the arm curriculum (ablation) | ALMI · CMU, hard | E_vel (m/s): 0.3658 -> 0.2202 | Helped |
| Add the arm curriculum (ablation) | ALMI · CMU, medium | Survival: 0.9652 -> 0.9852 | Helped |
| Train separate upper- and lower-body policies with adversarial iteration rather than one whole-body policy | ALMI · CMU, hard | E_vel (m/s): 0.3178 -> 0.2202 | Helped |
| Train separate upper- and lower-body policies with adversarial iteration rather than one whole-body policy | ALMI · CMU, hard | E_upper_jpe (m): 0.7022 -> 0.2116 | Helped |
| Use ALMI rather than Exbody | ALMI · CMU, hard | Survival: 0.8778 -> 0.9723 | Helped |
| Train on the complete ALMI-X dataset rather than a waving-related subset | Transformer foundation model · MuJoCo language-guided whole-body control | performance | Hurt |
Surprising findings
- Foundation model performance degraded when trained on the full ALMI-X dataset compared with a waving-only subset, indicating limits on handling diverse behaviors.
- Training the upper-body adversary directly without constraints led to early termination or self-collision, motivating the fixed-policy sampling curriculum.
Limitations
- Performs suboptimally in highly dynamic whole-body tasks such as dancing, per the authors.
- Foundation model is exploratory; performance degrades on the full diverse dataset.
- Upper body only tracks motions without a unified task reward, so expressiveness is reduced.
Experimental setup
Datasets: AMASS CMU MoCap ALMI-X
Benchmarks: CMU MoCap (1122 clips, Isaac Gym evaluation) MuJoCo sim-to-sim Unitree H1-2 real-world tests
Metrics: mean linear velocity error mean angular velocity error upper-body joint position error upper-body keypoint error joint action difference projected gravity survival rate
Baselines: Exbody ExBody2 OmniH2O ALMI (whole) ALMI (w/o curriculum) ALMI (w/o adv. learning)
Builds on: PPO PHC AMASS Exbody
Closest papers by meaning
- HumanoidArena: Benchmarking Egocentric Hierarchical Whole-body Learning (NeurIPS 2026 · similarity 0.90)
- Hierarchical World Models as Visual Whole-Body Humanoid Controllers (ICLR 2025 · similarity 0.89)
- Universal Humanoid Motion Representations for Physics-Based Control (ICLR 2024 · similarity 0.88)
- SUGAR: A Scalable Human-Video-Driven Generalizable Humanoid Loco-Manipulation Learning Framework (NeurIPS 2026 · similarity 0.88)
- Humanoid Horizon: Extending Task Horizon in Whole-Body Loco-Manipulation via Parallel Training, Dynamic Starting, and Reward Gating (NeurIPS 2026 · similarity 0.88)
- KungfuBot: Physics-Based Humanoid Whole-Body Control for Learning Highly-Dynamic Skills (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.