Research map / Label-efficient and robust learning / Recognition and long-tail
From Experts to a Generalist: Toward General Whole-Body Control for Humanoid Robots
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BumbleBee, an expert-to-generalist pipeline that clusters AMASS motions with a text+kinematics autoencoder, trains cluster-specific tracking experts with iterative per-cluster delta-action sim-to-real refinement, and distills them into a Transformer generalist for the Unitree G1.
Problem
General agile whole-body motion tracking on humanoid robots suffers from conflicting gradients and mismatched data distributions across diverse motion types, and sim-to-real gaps hinder deployment.
Approach
Cluster the motion dataset via an autoencoder over kinematic features (including leg/foot features) and text embeddings, train expert tracking policies and per-cluster delta-action models, then distill all experts into one Gated Transformer-XL generalist with DAgger.
Key results
- BumbleBee reaches 66.84% success rate in MuJoCo versus 50.19% for Exbody2 and below 40% for OmniH2O and Hover.
- In IsaacGym BB achieves 89.58% SR, versus 86.63% for Exbody2.
- Iterative delta fine-tuning raises expert mean MuJoCo success from 51.49% (Iter 0) to 70.37% (Iter 2).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use the BumbleBee expert-to-generalist framework instead of Exbody2 | Unitree G1 whole-body control · Filtered AMASS, MuJoCo | SR: 50.19% -> 66.84% | Helped |
| use the BumbleBee expert-to-generalist framework instead of Exbody2 | Unitree G1 whole-body control · Filtered AMASS, IsaacGym | MPJPE: 0.2937 -> 0.1907 | Helped |
| train a generalist without expert specialization (ablation) | BumbleBee · Filtered AMASS, MuJoCo | SR: 66.84% -> 33.01% | Hurt |
| partition training data into six random subsets rather than meaningful motion clusters (ablation) | BumbleBee · Filtered AMASS, MuJoCo | SR: 66.84% -> 35.36% | Hurt |
| apply one iteration of delta-action refinement (ablation) | expert tracking policies · Experts on respective motion clusters, MuJoCo | SR: 51.49% -> 60.33% | Helped |
| apply a second iteration of delta-action refinement (ablation) | expert tracking policies · Experts on respective motion clusters, MuJoCo | SR: 60.33% -> 70.37% | Helped |
| replace cluster-specific delta action models with one general delta action model (ablation) | expert tracking policies · Jump cluster, MuJoCo | SR: 59.64% -> 50.71% | Hurt |
| replace cluster-specific delta action models with one general delta action model (ablation) | expert tracking policies · Stand-up cluster, MuJoCo | SR: 64.11% -> 75.85% -> 77.32% | Hurt |
| replace cluster-specific delta action models with one general delta action model (ablation) | expert tracking policies · Walk-slow cluster, MuJoCo | SR: 15.71% -> 42.32% -> 56.50% | Hurt |
Surprising findings
- A single general delta action model struggles especially on the Jump cluster, with Expert Gen Final dropping to 50.71% versus 59.64% for Expert Init.
- Random partitioning of data into six subsets gives no clear advantage over the general initialization (35.36% vs 33.01% in MuJoCo).
Limitations
- No global localization sensors (GPS/VIO), causing biases when aligning to reference motions.
- Pipeline complexity limits scalability, especially when integrating real-world training feedback.
- Iterative refinement limited to two iterations due to computational resources.
Experimental setup
Datasets: AMASS HumanML3D real-world G1 trajectories
Benchmarks: IsaacGym MuJoCo
Metrics: Success Rate MPJPE MPKPE
Baselines: OmniH2O Exbody2 Hover General Init Random clustering
Builds on: ASAP PHC DAgger HumanML3D Exbody2
Closest papers by meaning
- HumanoidArena: Benchmarking Egocentric Hierarchical Whole-body Learning (NeurIPS 2026 · similarity 0.88)
- Hierarchical World Models as Visual Whole-Body Humanoid Controllers (ICLR 2025 · similarity 0.87)
- H-GAP: Humanoid Control with a Generalist Planner (ICLR 2024 · similarity 0.87)
- SUGAR: A Scalable Human-Video-Driven Generalizable Humanoid Loco-Manipulation Learning Framework (NeurIPS 2026 · similarity 0.87)
- Universal Humanoid Motion Representations for Physics-Based Control (ICLR 2024 · similarity 0.85)
- Adversarial Locomotion and Motion Imitation for Humanoid Policy Learning (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.