Research map / Probabilistic and scientific modeling / Dynamical systems

Hybrid Internal Model: Learning Agile Legged Locomotion with Simulated Robot Response

ICLR 2024 · Probabilistic and scientific modeling › Dynamical systems · approach: Latent-space control and planning · 4 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Hybrid Internal Model (HIM), which estimates external environment disturbances from the robot's proprioceptive response via contrastive (SwAV-style) learning, enabling a proprioception-only locomotion policy trained with PPO that transfers to real quadrupeds.

Problem

Legged robot locomotion requires accurate estimates of external states (terrain friction, elevation maps) that onboard proprioceptive sensors cannot directly provide; existing two-stage teacher-student or adaptation methods suffer from information loss and noise sensitivity in regression-based estim

Approach

Treat external environment states as disturbances and estimate them from the robot's response embedded in its successor state, using a hybrid embedding (explicit velocity plus implicit stability latent) optimized by prototype-based contrastive learning (SwAV-style with Sinkhorn-Knopp) over batches of trajectories, trained jointly with PPO in a single stage without privileged inputs to the policy.

Key results

• Real-world short-range stairs success rate 100% for HIM vs 60% for RMA and 0% for MoB and built-in MPC.
• Real-world long-range stairs on Aliengo: 176.5 stairs ascended vs 75.35 for RMA; compositional terrain success 85% vs 45% for RMA.
• Simulation tracking errors on most slope/stairs/obstacle tasks are lowest for HIM (e.g., stairs linear [-1,1] m/s: 0.147 vs 0.306 for RMA).

Measured effects reported in the paper

Surprising findings

• Ablation with a regression-based hybrid internal model performs close to contrastive HIM on some real-world tasks, but on long-range stairs and missing-step tests the contrastive version is clearly better.
• Increasing the number of prototypes improves performance up to 64 but then degrades final performance, suggesting sparse representations are inadequate yet overly dense ones also hurt.

Limitations

• No external perception (vision, lidar, elevation sensing), leaving the robot with a limited skill set.
• Real-world success rates on hard tasks remain moderate (e.g., 55% on deformable slope, 85% on compositional terrain).
• Choice of prototype count is heuristic and requires further study.

Experimental setup

Datasets: Isaac Gym simulated terrains (slopes, rough slopes, stairs, discrete obstacles)

Benchmarks: real-world stairs, unseen terrains, and anti-disturbance benchmarks Isaac Gym tracking-error benchmarks on slopes, rough slopes, stairs, discrete obstacles

Metrics: success rate maximum weight (kg) number of stairs tracking error normalized linear/angular velocity tracking score maximum reachable terrain level

Baselines: Rapid Motor Adaptation Walk These Ways (MoB) Built-in MPC Regression Baseline (PPO without HIM) Oracle

Builds on: Proximal Policy Optimization SwAV Rapid Motor Adaptation Walk These Ways

Closest papers by meaning

• Learning Vision-Guided Quadrupedal Locomotion End-to-End with Cross-Modal Transformers (ICLR 2022 · similarity 0.88)
• Adversarial Locomotion and Motion Imitation for Humanoid Policy Learning (NeurIPS 2025 · similarity 0.86)
• ReinFlow: Fine-tuning Flow Matching Policy with Online Reinforcement Learning (NeurIPS 2025 · similarity 0.85)
• CrossLoco: Human Motion Driven Control of Legged Robots via Guided Unsupervised Reinforcement Learning (ICLR 2024 · similarity 0.84)
• Let Humanoids Hike! Integrative Skill Development on Complex Trails (CVPR 2025 · similarity 0.84)
• Efficient Test-Time Adaptation For Robot Policies (NeurIPS 2026 · similarity 0.84)