Research map / Reinforcement learning / Visual and representation RL
H-GAP: Humanoid Control with a Generalist Planner
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Trains H-GAP, a VQ-VAE plus transformer prior over state-action trajectories from MoCapAct, and uses it zero-shot via MPC planning for humanoid control tasks.
Problem
Learning general-purpose 56-DoF humanoid control from motion-capture-derived trajectories without online interaction, and transferring learned motor priors to novel downstream tasks.
Approach
A state-conditioned generative trajectory model that discretizes state-action sequences with a VQ-VAE (multiple codes per step) and models latent codes with an autoregressive transformer prior, then plans with top-p, high-temperature sampling and MPC selection by rollout reward, requiring no task-specific training.
Key results
- On six downstream tasks, H-GAP achieves a mean return of 46.18 versus 43.8 for TAP and 12.50 for MPPI, which has access to the ground-truth model.
- In imitation, H-GAP reaches 46.02 ± 1.57 returns versus 29.38 ± 2.43 for an MLP behaviour cloning baseline.
- Larger H-GAP models (up to 300M) improve validation loss and imitation but do not consistently improve downstream task performance, with lower latent-code entropy at larger sizes.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use H-GAP instead of MLP BC | MoCap imitation · MoCap imitation | Returns: 29.38 ± 2.43 -> 46.02 ± 1.57 | Helped |
| use H-GAP instead of the strongest baseline for each task | downstream control · speed, rotate y, jump | return: 65.5 ± 2.3 -> 97.45 ± 3.81; 60.2 ± 12.5 -> 103.33 ± 5.61; 3.3 ± 0.2 -> 3.71 ± 0.35 | Helped |
| use H-GAP instead of the strongest baseline for each task | downstream control · forward, shift left, backward | return: 121.2 ± 10.9 -> 70.06 ± 4.83; 27.1 ± 1.0 -> 5.33 ± 2.74; 11.6 ± 9.9 -> -2.81 ± 4.14 | Hurt |
| replace MPPI with H-GAP planning | MPC generalist · downstream tasks | return: 36.7 ± 1.2 -> 97.45 ± 3.81; 23.9 ± 3.3 -> 103.33 ± 5.61; 0.1 ± 0.0 -> 3.71 ± 0.35; -0.9 ± 2.2 -> 70.06 ± 4.83; 4.4 ± 0.9 | Mixed |
| add a naive Gaussian action prior to MPPI | MPPI · speed, forward, jump versus other downstream tasks | return: 36.7 ± 1.2 -> 55.1 ± 1.4; -0.9 ± 2.2 -> 18.1 ± 3.6; 0.1 ± 0.0 -> 2.0 ± 0.2 | Mixed |
| use 8 codes per step instead of 16 (ablation) | H-GAP · downstream tasks (mean) | return: 35.90 ± 1.18 -> 32.84 ± 1.29 | Hurt |
| use 32 codes per step instead of 16 (ablation) | H-GAP · downstream tasks (mean) | return: 35.90 ± 1.18 -> 32.80 ± 0.85 | Hurt |
| increase training sequence length from 16 to 24 while keeping horizon 16 (ablation) | H-GAP · downstream tasks (mean) | return: 35.90 ± 1.18 -> 32.86 ± 1.73 | Hurt |
| increase training sequence length from 16 to 32 while keeping horizon 16 (ablation) | H-GAP · downstream tasks (mean) | return: 35.90 ± 1.18 -> 28.55 ± 1.61 | Hurt |
| increase training sequence length from 16 to 64 while keeping horizon 16 (ablation) | H-GAP · downstream tasks (mean) | return: 35.90 ± 1.18 -> 23.20 ± 1.43 | Hurt |
| reduce the number of planning samples from 256 to 128 at top p 0.99 and temperature 4 (ablation) | H-GAP · downstream tasks | return: 38.81 -> 37.29 | Hurt |
| reduce top p from 0.99 to 0.98 at temperature 4 and 256 samples (ablation) | H-GAP · downstream tasks | return: 38.81 -> 36.39 | Hurt |
Surprising findings
- Scaling model size from 6M to 300M degrades downstream control performance despite better imitation and validation loss.
- CRR and IQL perform poorly on relabelled MoCapAct tasks, while TD3-BC is stronger than them.
Limitations
- Performance may dip in sparse-reward settings requiring long-term planning.
- Less effective in partially observable or stochastic environments.
- Scaling model size saturates downstream performance given current dataset.
Experimental setup
Datasets: MoCapAct CMU MoCap
Benchmarks: MoCapAct imitation tracking six downstream humanoid control tasks (speed, rotate y, jump, forward, shift left, backward)
Metrics: episodic return episode length state prediction error latent code entropy
Baselines: CRR IQL TD3-BC TAP MPPI MPPI_{mu,sigma} MLP BC
Builds on: Trajectory Autoencoding Planner VQ-VAE MPPI
Closest papers by meaning
- Universal Humanoid Motion Representations for Physics-Based Control (ICLR 2024 · similarity 0.89)
- Humanoid Horizon: Extending Task Horizon in Whole-Body Loco-Manipulation via Parallel Training, Dynamic Starting, and Reward Gating (NeurIPS 2026 · similarity 0.88)
- SUGAR: A Scalable Human-Video-Driven Generalizable Humanoid Loco-Manipulation Learning Framework (NeurIPS 2026 · similarity 0.88)
- Hierarchical World Models as Visual Whole-Body Humanoid Controllers (ICLR 2025 · similarity 0.88)
- HumanoidArena: Benchmarking Egocentric Hierarchical Whole-body Learning (NeurIPS 2026 · similarity 0.88)
- Humanoid Locomotion as Next Token Prediction (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.