Research map / 3D vision / Depth and stereo

H-Flow: Self-supervised Human Scene Flow via Physics-inspired Joint Multi-modal Learning

NeurIPS 2026 · 3D vision › Depth and stereo · approach: Learned optical and scene flow · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes H-Flow, a self-supervised multi-head transformer that jointly estimates dense human scene flow, pose, depth, mask and camera from monocular video using physics-inspired cross-modal constraints, and introduces the synthetic DynAct4D benchmark with vertex-level flow ground truth.

Problem

Dense 3D scene flow for articulated, clothed humans lacks 4D ground truth, and generic scene flow methods rely on rigid-body assumptions that fail on non-rigid human motion; parametric models miss surface dynamics.

Approach

Jointly predict depth, flow, mask, pose and camera from a shared transformer and supervise flow without labels using four physics-inspired constraints (silhouette edge alignment, skeletal-surface coupling, center-of-mass support, minimum-jerk effort path) plus a distillation anchor to teachers for metric scale.

Key results

• On Fit3D, H-Flow achieves 24.2 mm EPE versus 38.3 mm for the best competitor (Depth Pro + H-MoRe), a 37% reduction.
• On out-of-domain DynAct4D, H-Flow reaches 28.2 mm EPE versus 42.2 mm for Depth Pro + H-MoRe.
• Depth MAE on Fit3D body region drops from 234.3 mm (Depth Pro) to 124.5 mm (H-Flow).

Surprising findings

• Removing the distillation anchor causes absolute errors to double or triple while scale-invariant metrics stay roughly unchanged.
• Removing the center-of-mass constraint slightly improves Fit3D flow EPE (26.2 vs 24.2 is worse; the ablation shows w/o CoM is 26.2, still higher than full model).

Limitations

• Physical priors are scale-invariant, so global metric scale depends on a distillation anchor to noisy teachers (Depth Pro, SAM 3D Body).
• Pose head lags on PA-MPJPE versus 2D-keypoint-based lifting methods such as HiPART and MotionAGFormer.
• Evaluation of DynAct4D is only on synthetic data; code, models, and benchmark are promised only upon publication.

Experimental setup

Datasets: Human3.6M Fit3D 3DPW

Benchmarks: Fit3D DynAct4D 3DPW

Metrics: EPE 1-Cos Acc.S Acc.R MPJPE PA-MPJPE MAE SiLog

Baselines: ZeroMSF Self-Mono-SF Depth Pro + H-MoRe Depth Pro + RAFT SAM 3D Body WHAM HiPART MotionAGFormer MotionBERT CameraHMR PromptHMR Depth Anything V2

Builds on: DINOv3 DPT SAM minimum-jerk principle SMPL-X

Closest papers by meaning

• H-MoRe: Learning Human-centric Motion Representation for Action Analysis (CVPR 2025 · similarity 0.90)
• Flow-NeRF: Joint Learning of Geometry, Poses, and Dense Flow within Unified Neural Representations (CVPR 2025 · similarity 0.87)
• HuPerFlow: A Comprehensive Benchmark for Human vs. Machine Motion Estimation Comparison (CVPR 2025 · similarity 0.86)
• E-MoFlow: Learning Egomotion and Optical Flow from Event Data via Implicit Regularization (NeurIPS 2025 · similarity 0.86)
• Motions as Queries: One-Stage Multi-Person Holistic Human Motion Capture (CVPR 2025 · similarity 0.86)
• Zero-Shot Monocular Scene Flow Estimation in the Wild (CVPR 2025 · similarity 0.86)