Research map / 3D vision / Face and shape
Thin-Shell-SfT: Fine-Grained Monocular Non-rigid 3D Surface Tracking with Neural Deformation Fields
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Thin-Shell-SfT tracks highly deformable cloth from a monocular static-camera video by representing the surface as continuous neural deformation fields with a Kirchhoff-Love thin-shell physics prior and surface-induced 3D Gaussian splatting.
Problem
Fine-grained non-rigid 3D tracking of highly deformable surfaces (e.g. cloth wrinkles) from monocular RGB video is ill-posed; prior methods use discrete meshes/points, frame-by-frame optimisation, and mesh-based renderers that fail to recover fine details.
Approach
Replace discrete meshes with coordinate-based sine-MLP neural deformation fields over a continuous parametric domain, impose a continuous Kirchhoff-Love hyperelastic strain-energy prior at resampled points, optimise all frames jointly with causal deformation parameterisation, and render with 3D Gaussians fixed to the template's local frame.
Key results
- Average Chamfer distance 3.3 (x1e-4) over nine phi-SfT sequences vs 3.93 for phi-SfT, the previous best.
- Normal error 0.009 (l2) and 0.034 (cosine) vs 0.013 and 0.041 for phi-SfT.
- Ablation: removing physics prior raises average Chamfer distance to 34.25 vs 3.46 for the full model.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Thin-Shell-SfT instead of ϕ-SfT | ϕ-SfT dataset · ϕ-SfT dataset, Avg | Chamfer distance (×10^4): 3.93 -> 3.3†/3.5 | Helped |
| add the thin-shell physics prior (ablation) | Thin-Shell-SfT · ϕ-SfT dataset | Chamfer distance (×10^4): 34.25 -> 3.46 | Helped |
| add surface induction of Gaussians (ablation) | Thin-Shell-SfT · ϕ-SfT dataset | Chamfer distance (×10^4): 14.0 -> 3.46 | Helped |
| fix the Gaussian normal scale (ablation) | Thin-Shell-SfT · ϕ-SfT dataset | Chamfer distance (×10^4): 3.75 -> 3.46 | Helped |
| optimise randomly selected frames instead of all frames jointly (ablation) | Thin-Shell-SfT · ϕ-SfT dataset | reconstruction quality | Hurt |
Surprising findings
- Dynamic view synthesis methods (K-Planes, Deformable Gaussians) recover texture rather than accurate geometry in monocular static-camera settings.
Limitations
- Struggles with extreme self-collisions.
- Textureless surfaces are not handled.
- Requires a template (first frame) and per-sequence optimisation of networks.
Experimental setup
Datasets: phi-SfT dataset
Benchmarks: phi-SfT dataset
Metrics: Chamfer distance normal error (l2) cosine normal consistency
Baselines: phi-SfT Stotko et al. Ngo2015 DDD Shimada IsMo-GAN N-NRSfM Diff-NRSfM K-Planes Deformable Gaussians
Builds on: 3D Gaussian Splatting NeuralClothSim phi-SfT SIREN Kirchhoff-Love thin shell theory
Closest papers by meaning
- NeuralClothSim: Neural Deformation Fields Meet the Thin Shell Theory (NeurIPS 2024 · similarity 0.88)
- Physics-guided Shape-from-Template: Monocular Video Perception through Neural Surrogate Models (CVPR 2024 · similarity 0.87)
- MorpheuS: Neural Dynamic 360deg Surface Reconstruction from Monocular RGB-D Video (CVPR 2024 · similarity 0.87)
- DGS-LRM: Real-Time Deformable 3D Gaussian Reconstruction From Monocular Videos (NeurIPS 2025 · similarity 0.85)
- Motion2VecSets: 4D Latent Vector Set Diffusion for Non-rigid Shape Reconstruction and Tracking (CVPR 2024 · similarity 0.85)
- Gaussian-Flow: 4D Reconstruction with Dynamic 3D Gaussian Particle (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.