Research map / Reinforcement learning / Visual and representation RL
Zero-Shot Whole-Body Humanoid Control via Behavioral Foundation Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces FB-CPR, an online unsupervised RL method that regularizes forward-backward zero-shot policy learning with imitation of unlabeled AMASS motions, yielding the humanoid behavioral foundation model Meta Motivo for zero-shot reward, goal, and tracking tasks.
Problem
Zero-shot whole-body control of high-dimensional, unstable humanoid agents: existing zero-shot unsupervised RL performs poorly without good data coverage and provides weak inductive bias, while imitation-based sequence models and behavior-prior methods need per-task RL or fail to generalize.
Approach
Ground forward-backward (FB) representations with a conditional distribution-matching regularizer: trajectories from unlabeled motion capture are embedded into the FB latent space via the average backward embedding, and a latent-conditioned discriminator plus critic pushes the online policy to cover those motions while the FB value term keeps zero-shot promptability.
Key results
- FB-CPR reaches 61% of TD3 (single-task) reward performance and 151.68 vs ASE 105.73 average reward; it outperforms ASE by over 1.4x in each category.
- Tracking success of 0.83 on train and 0.83 on test for zero-shot FB-CPR, versus 0.40 test for ASE and 0.73 test for CALM.
- A 288M-parameter transformer-style FB-CPR scales reward score_norm to 0.72 and tracking test success to 0.89 (score_norm).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use latent-conditioned rather than state-only discriminator (ablation) | FB-CPR · Humanoid tracking | tracking performance | Helped |
| use latent-conditioned rather than state-only discriminator (ablation) | FB-CPR · Humanoid reward tasks | return | No clear effect |
| include the unsupervised value-function term F(s,a,z)^T z in the actor loss (ablation) | FB-CPR · Humanoid reward tasks | return | Helped |
| include the unsupervised value-function term F(s,a,z)^T z in the actor loss (ablation) | FB-CPR · Humanoid tracking | tracking performance | Helped |
| train online with conditional policy regularization rather than offline with advantage-weighted actor updates (ablation) | FB-CPR · Humanoid tasks | overall performance | Helped |
| increase model capacity together with the number of demonstrations (ablation) | FB-CPR · Humanoid tracking | tracking performance | Helped |
| replace MLP layers with residual, layer-normalized Mish blocks and scale from ~25M to ~288M parameters (ablation) | FB-CPR · Humanoid tasks | overall performance | Helped |
Surprising findings
- Human evaluators judged FB-CPR more human-like in 83% of reward tasks despite TD3 achieving higher returns.
- Offline advantage-weighted FB trained on a large dataset performs substantially worse than FB-CPR trained online with policy regularization.
Limitations
- Struggles with motions far from the mocap distribution, such as ground-movement reward tasks, and produces unnatural falling or standing motions.
- Limited to proprioceptive observations; cannot solve navigation or object-interaction tasks.
- Relies on expensive motion capture datasets and lacks theoretical understanding of its components.
Experimental setup
Datasets: AMASS AMASS-Act
Benchmarks: 45 reward tasks (55 evaluated) 50 goal poses AMASS tracking train/test split
Metrics: average return goal proximity goal success rate tracking success earth mover's distance
Baselines: TD3 Goal-TD3 Goal-GAIL PHC CALM ASE MPPI Diffuser H-GAP FB-AW
Builds on: forward-backward representations ASE CALM TD3 Goal-GAIL
Closest papers by meaning
- Foundation Policies with Hilbert Representations (ICML 2024 · similarity 0.88)
- Hierarchical World Models as Visual Whole-Body Humanoid Controllers (ICLR 2025 · similarity 0.87)
- Universal Humanoid Motion Representations for Physics-Based Control (ICLR 2024 · similarity 0.87)
- Humanoid Horizon: Extending Task Horizon in Whole-Body Loco-Manipulation via Parallel Training, Dynamic Starting, and Reward Gating (NeurIPS 2026 · similarity 0.86)
- SUGAR: A Scalable Human-Video-Driven Generalizable Humanoid Loco-Manipulation Learning Framework (NeurIPS 2026 · similarity 0.86)
- HumanoidArena: Benchmarking Egocentric Hierarchical Whole-body Learning (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.