Research map / Vision-language and multimodal / Robot action and navigation
Let Humanoids Hike! Integrative Skill Development on Complex Trails
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LEGO-H, an end-to-end vision-based hierarchical RL framework (TC-ViTs navigation plus VAE-based hierarchical metric loss for privileged distillation) that lets simulated Unitree H1 and G1 humanoids hike complex trails.
Problem
Humanoid robots lack a unified framework that integrates low-level motor skills with high-level, perception-aware navigation needed for independent hiking on complex, unpredictable trails.
Approach
A Temporal Vision Transformer (TC-ViTs) anticipates near-future local navigation goals and softly guides a locomotion policy, while a VAE with masking produces a hierarchical loss set that transfers joint-level action structure from a privileged teacher to a vision-based student.
Key results
- LEGO-H achieves 68.40% success rate on H1 hiking, versus 65.17% for RMA-H, 48.11% for RMA-B and 28.80% for EP-H.
- Removing the hierarchical loss metric set drops success rate from 68.40% to 64.73% and raises MEV from 7.84% to 10.40%.
- Vanilla ConvGRU navigation variant reaches only 42.97% success rate, far below TC-ViTs.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the Hierarchical Loss Metric set (ablation) | LEGO-H · H1 hiking trails | Success Rate (%): 64.73% to 68.40% | Helped |
| Add the Hierarchical Loss Metric set (ablation) | LEGO-H · H1 hiking trails | MEV (%): 10.40% to 7.84% | Helped |
| Add the Hierarchical Loss Metric set (ablation) | LEGO-H · H1 hiking trails | TTF (s): 7.00s to 7.46s | Helped |
| Add the Hierarchical Loss Metric set (ablation) | LEGO-H · H1 hiking trails | T2R (s): 5.13s to 4.95s | Helped |
| Replace TC-ViTs with ConvGRU (ablation) | LEGO-H · H1 hiking trails | Success Rate (%): 68.40 ± 1.34 -> 42.97 ± 0.67 | Hurt |
| Replace TC-ViTs with ConvGRU (ablation) | LEGO-H · H1 hiking trails | MEV (%): 7.84 ± 0.92 -> 9.41 ± 1.27 | Mixed |
| Use visual perception instead of blind inputs | RMA-H vs RMA-B · H1 hiking trails | Success Rate (%): 48.11 ± 0.72 -> 65.17 ± 2.05 | Helped |
| Use TC-ViTs goal-aligned, multi-scale visual processing instead of independent depth frames | LEGO-H vs EP-H · H1 hiking trails | Success Rate (%): 28.80 ± 0.88 -> 68.40 ± 1.34 | Helped |
| Use unified cross-level navigation and locomotion learning | LEGO-H vs RMA-H · H1 hiking trails | Success Rate (%): 65.17 ± 2.05 -> 68.40 ± 1.34 | Helped |
Surprising findings
- LEGO-H matches or slightly exceeds the privileged oracle policy on efficiency and safety metrics despite lower success rate.
- RMA-H has slightly higher traverse rate (74.61%) than LEGO-H (71.96%).
Limitations
- Upper-body pose is frozen, so whole-body control with hands for contact is not studied.
- Evaluation is simulation-only; real-world hardware results are not reported in the main text.
- Success rate remains around 68% and trail completion around 53%, below the oracle on completion metrics.
Experimental setup
Benchmarks: Simulated humanoid hiking trails (5 trail types x 5 difficulty levels)
Metrics: Success Rate Trail Completion Traverse Rate MEV TTF Time-to-Reach
Baselines: EP-H RMA-H RMA-B Oracle policy Vanilla (ConvGRU navigation)
Builds on: ViViT RMA Efficient Perceptive (EP) locomotion PPO DAgger
Closest papers by meaning
- Humanoid Horizon: Extending Task Horizon in Whole-Body Loco-Manipulation via Parallel Training, Dynamic Starting, and Reward Gating (NeurIPS 2026 · similarity 0.85)
- Hybrid Internal Model: Learning Agile Legged Locomotion with Simulated Robot Response (ICLR 2024 · similarity 0.84)
- HumanoidArena: Benchmarking Egocentric Hierarchical Whole-body Learning (NeurIPS 2026 · similarity 0.84)
- Humanoid Locomotion as Next Token Prediction (NeurIPS 2024 · similarity 0.83)
- SUGAR: A Scalable Human-Video-Driven Generalizable Humanoid Loco-Manipulation Learning Framework (NeurIPS 2026 · similarity 0.83)
- Hierarchical World Models as Visual Whole-Body Humanoid Controllers (ICLR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.