Research map / 3D vision / Human pose estimation
Multi-View 3D Human Pose Estimation with Weakly Synchronized Images
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces multi-view 3D human pose estimation from weakly synchronized images, releases the WeakSyncPose3D dataset (~800k frames), and proposes SyncDiffPose, a diffusion-based denoiser over multi-view 2D poses.
Problem
Multi-view 3D human pose estimation normally requires hardware-synchronized cameras; when views are only weakly synchronized (time offsets within one frame), fast-moving joints get corrupted, and no dataset or method existed for this setting.
Approach
Treat the error from intra-frame synchronization offsets as noise and use a diffusion model with a noise embedding module, conditioned on multi-view 2D poses, to denoise the 3D pose estimate; paired with a new weakly synchronized dataset labeled with OptiTrack mocap.
Key results
- On Human3.6M with frameshift-perturbed (unsynchronized) views, SyncDiffPose reaches 69.17 avg MPJPE vs 92.98 for MvP and 90.91 for Karim et al.
- On WeakSyncPose3D with ground-truth 2D poses, SyncDiffPose achieves 70.24 avg MPJPE vs 94.29 for MvP
- Ablation on Human3.6M-based setting: adding diffusion and NE module lowers avg error from 104.70 (MvP) to 84.86
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SyncDiffPose instead of MvP | 3D human pose estimation with weakly synchronized inputs · WeakSyncPose3D (ShuffleNet V2 2D poses) | MPJPE: 104.70 -> 84.86 | Helped |
| Use SyncDiffPose instead of MvP | 3D human pose estimation with weakly synchronized inputs · WeakSyncPose3D (ground-truth 2D poses) | MPJPE: 94.29 -> 70.24 | Helped |
| Use SyncDiffPose instead of MvP | 3D human pose estimation · Human3.6M (synchronized) | MPJPE: 32.97 -> 30.85 | Helped |
| Use SyncDiffPose instead of Karim et al. | 3D human pose estimation with frame-shifted views · Human3.6M (0-or-1-frame shift) | MPJPE: 90.91 -> 69.17 | Helped |
| Add diffusion (ablation) | MvP on WeakSyncPose3D · WeakSyncPose3D (ShuffleNet V2 2D poses) | MPJPE: 104.70 -> 103.41 | Helped |
| Add diffusion (ablation) | SyncPose on WeakSyncPose3D · WeakSyncPose3D (ShuffleNet V2 2D poses) | MPJPE: 115.98 -> 101.84 | Helped |
| Add the NE module to diffusion (ablation) | SyncPose on WeakSyncPose3D · WeakSyncPose3D (ShuffleNet V2 2D poses) | MPJPE: 101.84 -> 84.86 | Helped |
Surprising findings
- On synchronized Human3.6M, SyncDiffPose (30.85) is worse than MvP (32.97) on the synchronized split only in some joints; the paper's gains are mainly on weakly synchronized data
Limitations
- Performance remains below models trained on fully synchronized inputs
- Only frame-level (within one frame) weak synchronization is studied; inter-frame offsets handled by timer method
- Reported results use 3 views and 16 joints in a fixed setup
Experimental setup
Datasets: WeakSyncPose3D Human3.6M
Benchmarks: WeakSyncPose3D Human3.6M
Metrics: MPJPE
Baselines: Denis et al. (Tome et al. 2018) Karim et al. (Iskakov et al. 2019) MvP (Zhang et al. 2021)
Builds on: DDPM MvP ShuffleNet V2
Closest papers by meaning
- MV-SSM: Multi-View State Space Modeling for 3D Human Pose Estimation (CVPR 2025 · similarity 0.86)
- PoseIRM: Enhance 3D Human Pose Estimation on Unseen Camera Settings via Invariant Risk Minimization (CVPR 2024 · similarity 0.85)
- Advancing Video Synchronization with Fractional Frame Analysis: Introducing a Novel Dataset and Model (AAAI 2024 · similarity 0.85)
- $\text{Di}^2\text{Pose}$: Discrete Diffusion Model for Occluded 3D Human Pose Estimation (NeurIPS 2024 · similarity 0.84)
- SVTformer: Spatial-View-Temporal Transformer for Multi-View 3D Human Pose Estimation (AAAI 2025 · similarity 0.84)
- COMPOSE: Hypergraph Cover Optimization for Multi-view 3D Human Pose Estimation (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.