Research map / 3D vision / Face and shape
Non-Rigid Structure-from-Motion: Temporally-Smooth Procrustean Alignment and Spatially-Variant Deformation Modeling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a temporally-smooth Procrustean alignment (TPA) module and a spatial-weighted nuclear norm (SWNN) for non-rigid structure-from-motion, achieving better 3D reconstruction on MoCap, NRSfM Challenge, semi-dense and H3WB benchmarks.
Problem
Non-rigid structure-from-motion suffers from rotation/motion ambiguity and global low-rank priors that over-penalize drastic, spatially-variant deformations.
Approach
Align consecutive 3D shapes via a temporal smoothness loss (no mean reference shape) to correct camera rotations, and apply the low-rank nuclear norm to spatially-weighted proxy shapes built from DFT-based nearly-rigid region segmentation so that deforming regions get weaker low-rank constraints.
Key results
- On the MoCap benchmark, the proposed method achieves the lowest error on Drink (0.0031), Pickup (0.0126), Yoga (0.0109), Stretch (0.0114), and Shark (0.0258) among compared methods.
- On the NRSfM Challenge dataset, it achieves best results on Articulated (10.69 mm), Stretch (5.43 mm), and Tearing (10.77 mm).
- On H3WB, mean normalized error is 0.1857 versus 0.2054 for R-BMM, the best baseline.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the full spatial-weighted nuclear norm (SWNN) module (ablation) | Ours · MoCap: Drink, Pickup, Yoga, Stretch, Dance, Face; NRSfM Challenge: Articulated (circle), Articulated (flyby), Stretch (zigzag) | 3D reconstruction error: 0.0050 -> 0.0031; 0.0137 -> 0.0126; 0.0120 -> 0.0109; 0.0132 -> 0.0114; 0.0932 -> 0.0921; 0.0161 -> 0.0144; 3.5626 -> 2. | Helped |
| add TPA to correct camera motion initialization (ablation) | BMM initialization · Camera-motion perturbation test (σ = 0.1, 0.2, 0.5) | e3d: 0.0493 -> 0.0237; 0.0845 -> 0.0592; 0.1743 -> 0.1665 | Helped |
| use BMM+Smooth instead of BMM for camera motion (ablation) | BMM camera-motion initialization · Camera-motion perturbation test (σ = 0.1, 0.2, 0.5) | e3d: 0.0493 -> 0.0845; 0.0845 -> 0.1149; 0.1743 -> 0.1905 | Hurt |
| use TPA alignment instead of GPA alignment (ablation) | 3D shape-sequence alignment · Shape alignment test | low-rank and first-order smoothness properties | Helped |
| use Our(I) initialization instead of the standard Ours initialization (ablation) | Ours on semi-dense data · Kinect and Mat | e3d: 0.0356 -> 0.0161; 0.0238 -> 0.0182 | Helped |
| use Our(I) initialization instead of the standard Ours initialization (ablation) | Ours on semi-dense data · Rug | e3d: 0.0088 -> 0.0088 | No clear effect |
Limitations
- Trajectory segmentation is heuristic and depends on hyperparameters (alpha_r, m_f, K_s) tuned per dataset.
- Non-convex optimization requires good initialization of camera motion and shapes.
- Authors note need for more accurate trajectory segmentation and explicit modeling of complex deformation.
Experimental setup
Datasets: MoCap benchmark (Drink, Pickup, Yoga, Stretch, Dance, Face, Walking, Shark) NRSfM Challenge dataset Semi-dense dataset (Kinect paper, rug, table mat) H3WB
Benchmarks: MoCap NRSfM Challenge Semi-dense H3WB
Metrics: mean normalized 3D reconstruction error e3d reconstruction error in mm
Baselines: CSF1 CSF2 KSTA PND PMP CNS PR BMM R-BMM OPM AOW BP
Builds on: Dai et al. BMM low-rank factorization General Procrustean Analysis (PND) Organic Priors Method (OPM) Kumar rectification (R-BMM)
Closest papers by meaning
- Scalable Dense Non-Rigid Structure-From-Motion: A Grassmannian Perspective (CVPR 2018 · similarity 0.89)
- Deep Non-Rigid Structure-from-Motion Revisited: Canonicalization and Sequence Modeling (AAAI 2025 · similarity 0.88)
- Structure-from-Motion with a Non-Parametric Camera Model (CVPR 2025 · similarity 0.87)
- VGGSfM: Visual Geometry Grounded Deep Structure From Motion (CVPR 2024 · similarity 0.85)
- Light3R-SfM: Towards Feed-forward Structure-from-Motion (CVPR 2025 · similarity 0.85)
- AnyMap: Learning a General Camera Model for Structure-from-Motion with Unknown Distortion in Dynamic Scenes (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.