Research map / 3D vision / Face and shape
Neural Parametric Gaussians for Monocular Non-Rigid Object Reconstruction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Neural Parametric Gaussians, a two-stage method that fits a low-rank neural deformation point model as regularization and then optimizes 3D Gaussians anchored in local oriented volumes driven by that model, for monocular non-rigid novel view synthesis.
Problem
Monocular non-rigid object reconstruction is severely underconstrained, and existing methods fail to produce consistent high-quality novel views from viewpoints far from training views on realistic sequences with few multi-view cues.
Approach
Stage one learns a coarse point model as a low-rank combination of learnable point basis vectors with MLP-predicted coefficients, supervised by mask Chamfer, rigidity and RAFT optical flow losses; stage two attaches 3D Gaussians parameterized by barycentric weights to k-nearest-neighbor local volumes oriented by triangles of the coarse points, so Gaussians move with the template without per-Gaussi
Key results
- On Unbiased4D (cactus sequence), NPG reaches 22.348 PSNR, 0.905 SSIM and 0.095 LPIPS, outperforming HyperNeRF (18.56 PSNR) and other baselines.
- On the D-NeRF benchmark, NPG averages 33.92 PSNR and 0.98 SSIM, slightly above TiNeuVox (31.52) and HexPlane (29.81).
- NPG renders a frame in about 0.05 s (20 fps) at 400x400 on an NVIDIA A40, versus 2.15 s for TiNeuVox and 0.94 s for HexPlane.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Neural Parametric Gaussians instead of TiNeuVox | D-NeRF · D-NeRF (average) | PSNR: 31.52 -> 33.92 | Helped |
| use Neural Parametric Gaussians instead of TiNeuVox | D-NeRF · D-NeRF (average) | LPIPS: 0.04 -> 0.03 | Helped |
| use Neural Parametric Gaussians instead of HyperNeRF | Unbiased4D · Unbiased4D (synthetic cactus) | PSNR: 18.56 -> 22.348 | Helped |
| use Neural Parametric Gaussians instead of HyperNeRF | Unbiased4D · Unbiased4D (synthetic cactus) | LPIPS: 0.12 -> 0.095 | Helped |
| add rigidity loss (ablation) | NPG · Unbiased4D synthetic cactus | PSNR: 20.73 -> 22.348 | Helped |
| enable template fine-tuning during stage 2 (ablation) | NPG · Unbiased4D synthetic cactus | PSNR: 20.92 -> 22.348 | Helped |
| increase deformation basis size from K = 25 to K = 100 (ablation) | NPG · Unbiased4D synthetic cactus | PSNR: 22.348 -> 21.80 | Hurt |
| reduce deformation basis size from K = 25 to K = 10 (ablation) | NPG · Unbiased4D synthetic cactus | PSNR: 22.348 -> 16.63 | Hurt |
| reduce deformation basis size from K = 25 to K = 2 (ablation) | NPG · Unbiased4D synthetic cactus | PSNR: 22.348 -> 15.67 | Hurt |
Surprising findings
- Lower-rank deformation bases over-regularize and sharply reduce quality (K=2 gives 15.67 PSNR), while larger bases overfit, so an intermediate basis size (K=25) is optimal.
Limitations
- Fails on highly static cameras, where the template can collapse to a flat surface.
- Requires object masks and COLMAP camera poses; static ground planes need extra SAM masking.
- Performance on the realistic setting depends on sequence complexity (camera motion, lighting, speed, deformation extent).
Experimental setup
Datasets: D-NeRF Unbiased4D
Benchmarks: D-NeRF Unbiased4D
Metrics: PSNR SSIM LPIPS
Baselines: D-NeRF TiNeuVox HexPlane HyperNeRF Nerfies
Builds on: 3D Gaussian splatting KeyPointTransporter Hexplane RAFT D-NeRF
Closest papers by meaning
- Deformable 3D Gaussians for High-Fidelity Monocular Dynamic Scene Reconstruction (CVPR 2024 · similarity 0.90)
- DGS-LRM: Real-Time Deformable 3D Gaussian Reconstruction From Monocular Videos (NeurIPS 2025 · similarity 0.88)
- World from Motion: Generative Dynamic Gaussian Reconstruction from Monocular Video (NeurIPS 2026 · similarity 0.88)
- 4D3R: Motion-Aware Neural Reconstruction and Rendering of Dynamic Scenes from Monocular Videos (NeurIPS 2025 · similarity 0.87)
- REACTO: Reconstructing Articulated Objects from a Single Video (CVPR 2024 · similarity 0.87)
- Gaussian-Flow: 4D Reconstruction with Dynamic 3D Gaussian Particle (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.