Research map / 3D vision / Human pose estimation

DiffusionPoser: Real-time Human Motion Reconstruction From Arbitrary Sparse Sensors Using Autoregressive Diffusion

CVPR 2024 · 3D vision › Human pose estimation · approach: Physics and sensor-based motion capture · 23 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes DiffusionPoser, a single diffusion model that reconstructs whole-body motion in real time from arbitrary combinations of IMUs and pressure insoles, using autoregressive inpainting-based denoising guidance instead of conditioning.

Problem

Motion reconstruction from sparse body-worn sensors has been limited to fixed sensor configurations (typically six IMUs); users need flexibility to choose the number and placement of sensors for an activity without retraining, while staying accurate and real-time.

Approach

A single diffusion transformer denoiser is trained on AMASS-synthesized IMU data and used at inference with inpainting-based guidance (not conditioning) on whichever sensor features are observed, applied autoregressively frame-by-frame at 20 Hz so any sensor subset works without retraining.

Key results

• On TotalCaptureReal with six IMUs (pelvis, head, wrists, shanks), DiffusionPoser reports GA 14.4 degrees and JPE 6.1 cm, within 1.1 degrees and 1 cm of the best baseline, while PIP has the lowest jitter (1.1 vs 2.8).
• Using 3 sensors (pelvis and wrists) is optimal for whole-body GA; pelvis and thighs for leg kinematics; upper arms and pelvis for back kinematics.
• Reducing denoising steps from 30 to 5 raises GA from 14.4 to 16.0 degrees and jitter from 2.8 to 4.4; the model runs at 20 Hz with 45 ms reconstruction latency.

Measured effects reported in the paper

Surprising findings

• Adding noise to the initialization of the last frame (starting denoising from high noise) improved root translation error but worsened angular error and jitter.
• Smaller transformer (4 layers) gave slightly better root translation error than the larger model, though the authors state they do not know why.

Limitations

• Latency (about 45 ms reconstruction plus filtering) is too high for applications needing under 50 ms response.
• Sim-to-real error from calibration, sensor-to-bone motion, and synthetic-vs-real noise differences degrades accuracy with sparse sensors (e.g., 39.2 degrees GA for shanks only).
• Root translation estimation uses a heuristic contact-based correction that does not guarantee static contacts.

Experimental setup

Datasets: AMASS TotalCapture DIP-IMU OpenSim mocap datasets (three combined)

Benchmarks: TotalCaptureReal TotalCaptureSynth DIP-IMU

Metrics: Local Angular Error Global Angular Error Joint Position Error Jitter Root Translation Error

Baselines: Transpose PIP TIP

Builds on: DDPM DDIM MDM inpainting denoising (RePaint-style) PIP TIP

Closest papers by meaning

• Dynamic Inertial Poser (DynaIP): Part-Based Motion Dynamics Learning for Enhanced Human Pose Estimation with Sparse Inertial Sensors (CVPR 2024 · similarity 0.86)
• Spatial-Related Sensors Matters: 3D Human Motion Reconstruction Assisted with Textual Semantics (AAAI 2024 · similarity 0.86)
• REWIND: Real-Time Egocentric Whole-Body Motion Diffusion with Exemplar-Based Identity Conditioning (CVPR 2025 · similarity 0.86)
• UMotion: Uncertainty-driven Human Motion Estimation from Inertial and Ultra-wideband Units (CVPR 2025 · similarity 0.86)
• EnvPoser: Environment-aware Realistic Human Motion Estimation from Sparse Observations with Uncertainty Modeling (CVPR 2025 · similarity 0.86)
• ImDy: Human Inverse Dynamics from Imitated Observations (ICLR 2025 · similarity 0.85)