Research map / Reinforcement learning / Visual and representation RL

Humanoid Horizon: Extending Task Horizon in Whole-Body Loco-Manipulation via Parallel Training, Dynamic Starting, and Reward Gating

NeurIPS 2026 · Reinforcement learning › Visual and representation RL · approach: Dexterous manipulation transfer

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Humanoid Horizon, a single-policy RL framework for long-horizon multi-object humanoid loco-manipulation using parallel stage streams, dynamic starting from upstream terminal states, and reward gating, plus DAgger distillation into a VLA student.

Problem

Long-horizon whole-body humanoid loco-manipulation, where multiple large objects must be sequentially transported without resets; prior sequential or curriculum training suffers from easy-reward bias and catastrophic forgetting.

Approach

Jointly optimize all transport stages in one shared policy via parallel stage streams, initialize each stage from upstream rollout terminal states (Dynamic Starting), and zero later-stage reward when the preceding placed object is disturbed (Reward Gating).

Key results

• On 350 training tasks, Humanoid Horizon reaches 96.6% Succ1, 81.3% Succ2, and 80.4% SuccAll, versus 71.1% SuccAll for LHM-Humanoid.
• On 66 unseen tasks, SuccAll is 78.0% versus 61.6% for LHM-Humanoid.
• On five-object sequential transport, SuccAll is 60.2% versus 25.8% for LHM-Humanoid^dagger (five teachers).

Surprising findings

• Removing Dynamic Starting gives higher Succ1 (92.7%) than full model's 96.6% is not the case, but w/o DS yields a lower SuccAll (67.7%) while w/o RG yields Succ2 75.6% above full-model-free ablation on some metrics; the training-time SuccAll (87.6%) is higher than evaluation SuccAll (80.4%) because
• Training-time success is reported as independent per-stage streams, so it is not directly comparable to sequential evaluation SuccAll.

Limitations

• Succ1-to-Succ2 gap from cluttered post-task starting configurations and obstacle-aware navigation.
• Frozen-action grasp failures attributed to uniform scene allocation during training.
• Evaluated only in simulation (IsaacGym); real-world deployment is future work; VLA results are exploratory.

Experimental setup

Datasets: LHM-Humanoid benchmark (350 scenes)

Benchmarks: LHM-Humanoid 350 training tasks LHM-Humanoid 66 unseen tasks five-object sequential transport

Metrics: Succ1 Succ2 SuccAll Dist 1 Dist 2

Baselines: End-to-End RL Curriculum RL Hierarchical RL HumanVLA InterMimic TokenHSI LHM-Humanoid

Builds on: PPO AMP DAgger LHM-Humanoid

Closest papers by meaning

• SUGAR: A Scalable Human-Video-Driven Generalizable Humanoid Loco-Manipulation Learning Framework (NeurIPS 2026 · similarity 0.91)
• HumanoidArena: Benchmarking Egocentric Hierarchical Whole-body Learning (NeurIPS 2026 · similarity 0.90)
• Hierarchical World Models as Visual Whole-Body Humanoid Controllers (ICLR 2025 · similarity 0.89)
• Instruction-Augmented Long-Horizon Planning: Embedding Grounding Mechanisms in Embodied Mobile Manipulation (AAAI 2025 · similarity 0.88)
• Universal Humanoid Motion Representations for Physics-Based Control (ICLR 2024 · similarity 0.88)
• H-GAP: Humanoid Control with a Generalist Planner (ICLR 2024 · similarity 0.88)