Research map / Video / Video segmentation and tracking

StableHand: Quality-Aware Flow Matching for World-Space Dual-Hand Motion Estimation from Egocentric Video

NeurIPS 2026 · Video › Video segmentation and tracking · approach: Learned optical flow estimation · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

StableHand conditions a flow-matching DiT for world-space dual-hand motion on a learned per-component (wrist/finger, per-hand) quality signal, anchoring reliable observations and regenerating unreliable ones from a bimanual prior.

Problem

Recovering world-space 4D motion of two interacting hands from egocentric video, where hands leave view for long spans and are occluded by hand-object interaction, and existing methods condition uniformly on noisy per-frame observations.

Approach

A four-channel (wrist/finger × hand) quality signal, predicted by a learned quality network, drives a per-channel forward schedule, quality-adjusted velocity target, AdaLN modulation of a DiT, and quality-aware ODE initialization so that high-quality channels are preserved while low-quality ones are regenerated.

Key results

• On HOT3D, StableHand reaches W-MPJPE 57.83 mm vs 71.89 mm for HaWoR (about 20% reduction) and best on all four metrics.
• On ARCTIC, W-MPJPE 124.59 mm vs 171.61 mm for HaWoR (about 25% reduction), with best MRRPE 93.71 mm.
• Ablation on HOT3D: removing the quality schedule raises W-MPJPE from 57.83 to 92.32 mm; a single per-frame token raises it to 118.61 mm.

Surprising findings

• Adding the frozen WiLoR visual feature to the quality network degrades cross-corpus rank correlation, so excluding it helps.
• Removing the binary detection flag costs little in quality-network rank correlation, suggesting it is recoverable from the observation and its temporal context.

Limitations

• Quality signal is calibrated to a specific upstream estimator (WiLoR); swapping estimators requires retraining.
• Over long low-quality horizons the model may drift toward plausible but not faithfully grounded configurations.
• Object-hand joint modeling and end-to-end training are not addressed.

Experimental setup

Datasets: HOT3D ARCTIC H2O HOI4D Re:InterHand

Benchmarks: HOT3D ARCTIC

Metrics: PA-MPJPE W-MPJPE WA-MPJPE AccEr MRRPE

Baselines: HaMeR-SLAM WiLoR-SLAM HMP-SLAM Dyn-HaMR HaWoR UniHand

Builds on: conditional flow matching SDEdit RePaint MANO DiT

Closest papers by meaning

• HaWoR: World-Space Hand Motion Reconstruction from Egocentric Videos (CVPR 2025 · similarity 0.90)
• ComPose: When to Trust Hands for Object Pose Tracking (NeurIPS 2026 · similarity 0.90)
• HandEdit: A Unified Benchmark for Egocentric Human-to-Robot Dexterous Hand Image Editing (NeurIPS 2026 · similarity 0.89)
• Dyn-HaMR: Recovering 4D Interacting Hand Motion from a Dynamic Camera (CVPR 2025 · similarity 0.88)
• MEgoHand: Multimodal Egocentric Hand-Object Interaction Motion Generation (NeurIPS 2025 · similarity 0.88)
• The Surprising Effectiveness of Video Diffusion Models for Hand Motion Reconstruction (NeurIPS 2026 · similarity 0.88)