Research map / 3D vision / Face and shape
Link to the Past: Temporal Propagation for Fast 3D Human Reconstruction from Monocular Video
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TemPoFast3D, a plug-and-play frame-by-frame pipeline that propagates a canonical clothed-human shape via SMPL-based coordinate mapping and efficient sampling, reaching up to 12 FPS on monocular video.
Problem
Fast 3D clothed human reconstruction from monocular video: pixel-aligned single-image methods are slow per frame and video methods need minutes to hours of per-video optimization.
Approach
Performs shape inference in canonical space via SMPL-derived bidirectional coordinate mapping so that canonical shape is propagated across frames, and only queries visibility-guided, surface-adjacent points after an initial warm-up of n frames, wrapping existing pixel-aligned networks (GTA, SIFU) without retraining.
Key results
- Achieves a maximum of 12.3 FPS on the NeuMan citron sequence, a 3.06x speedup over the GTA baseline (3.27 FPS), with PSNR 31.13 vs 32.80.
- On NeuMan, TPF3D-GTA reaches PSNR 27.60 at 8.9 FPS, comparable to HumanNeRF and far faster than ExAvatar (4.0 FPS, 4h training).
- Three-view TPF3D-SIFU-3v improves THuman2.0 Chamfer to 0.4144 from 0.5961 for SIFU, while single-view variants slightly degrade quality.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TPF3D-SIFU-3v instead of SIFU† | SIFU · CAPE-NFP | Chamfer: 0.8263 -> 0.8024 | Helped |
| Use TPF3D-SIFU-3v instead of SIFU† | SIFU · CAPE-FP | Chamfer: 0.6254 -> 0.5794 | Helped |
| Use TPF3D-SIFU-3v instead of SIFU† | SIFU · THuman2.0 | Chamfer: 0.4409 -> 0.4144 | Helped |
| Use TPF3D-GTA-3v instead of GTA† | GTA · THuman2.0 | PSNR: 23.27 -> 23.21 | No clear effect |
| Add coordinate mapping (ablation) | GTA baseline pipeline · NeuMan, citron sequence | Max. FPS: 3.266 -> 2.142 | Hurt |
| Add a linear layer (ablation) | GTA with coordinate mapping · NeuMan, citron sequence | Max. FPS: 2.142 -> 2.669 | Helped |
| Add visibility-guided sampling (ablation) | GTA with linear layer · NeuMan, citron sequence | Max. FPS: 2.669 -> 1.908 | Hurt |
| Add surface-adjacent sampling (ablation) | GTA with visibility-guided sampling · NeuMan, citron sequence | Max. FPS: 1.908 -> 4.499 | Helped |
| Limit the number of sampling points (ablation) | GTA with surface-adjacent sampling · NeuMan, citron sequence | Max. FPS: 4.499 -> 5.841 | Helped |
| Add Torchscript optimization (ablation) | GTA with limited sampling points · NeuMan, citron sequence | Max. FPS: 5.841 -> 12.301 | Helped |
Surprising findings
- Single-view TPF3D variants slightly degrade reconstruction quality relative to the GTA and SIFU backbones they wrap, with gains appearing only when multiple views or frames are merged.
Limitations
- Occasional artifacts and sharp protrusions due to pose deformation
- Requires accurate alignment with SMPL, limiting reconstruction of loose clothing and accessories
- Reconstruction quality depends on accuracy of SMPL parameter estimation, which is hard for extreme poses and occlusions
Experimental setup
Datasets: THuman2.0 CAPE NeuMan
Benchmarks: THuman2.0 CAPE NeuMan
Metrics: Chamfer distance P2S normal error PSNR SSIM LPIPS FPS training time
Baselines: PIFu PIFuHD ECON GTA SIFU HumanNeRF InstantAvatar NeuMan Vid2Avatar GaussianAvatar 3DGS-Avatar ExAvatar
Builds on: PIFu GTA SIFU SMPL Vid2Avatar
Closest papers by meaning
- ODHSR: Online Dense 3D Reconstruction of Humans and Scenes from Monocular Videos (CVPR 2025 · similarity 0.89)
- D^3-Human: Dynamic Disentangled Digital Human from Monocular Video (CVPR 2025 · similarity 0.88)
- World from Motion: Generative Dynamic Gaussian Reconstruction from Monocular Video (NeurIPS 2026 · similarity 0.88)
- Guess The Unseen: Dynamic 3D Scene Reconstruction from Partial 2D Glimpses (CVPR 2024 · similarity 0.87)
- Beyond Static Features for Temporally Consistent 3D Human Pose and Shape From a Video (CVPR 2021 · similarity 0.87)
- MultiPly: Reconstruction of Multiple People from Monocular Video in the Wild (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.