Research map / 3D vision / Face and shape
PhyRecon: Physically Plausible Neural Scene Reconstruction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces PhyRecon, which couples differentiable volume rendering of SDF-based implicit surfaces with a differentiable particle-based physics simulator (via SPMC surface-point extraction) and joint rendering/physical uncertainty to produce physically stable indoor scene reconstructions.
Problem
Multi-view neural implicit reconstruction produces geometry that is physically implausible (unstable objects) and misses thin structures such as chair legs, because it relies only on rendering supervision and noisy monocular priors.
Approach
Jointly backpropagates rendering losses and a physical loss from a differentiable particle-based rigid-body simulator into SDF networks, using a fast differentiable surface-point extractor (SPMC) and a physical uncertainty grid that both reweights rendering losses and guides pixel sampling toward thin, unsupported structures.
Key results
- On ScanNet++ scene reconstruction, PhyRecon reaches CD 3.34 and F-Score 81.53 versus 3.87 / 78.45 for RICO.
- Object stability success rate (dropping simulation in Isaac Gym) rises to 78.16% on ScanNet++ versus 26.43% for RICO and 25.28% for ObjectSDF++.
- Authors report at least a 40% stability improvement across all datasets, and ablation shows physical loss alone raises ScanNet++ SR from 25.28% to 68.96%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add physical loss (ablation) | PhyRecon with no components enabled · ScanNet++ | Object stability SR (%): 25.28 -> 68.96 | Helped |
| add rendering uncertainty (ablation) | PhyRecon without other components · ScanNet++ | Scene CD: 3.82 -> 3.46 | Helped |
| add physical uncertainty to rendering uncertainty (ablation) | PhyRecon with rendering uncertainty · ScanNet++ | Object stability SR (%): 47.12 -> 56.32 | Helped |
| add physics-guided pixel sampling (ablation) | PhyRecon with rendering and physical uncertainty · ScanNet++ | Scene F-Score: 80.68 -> 81.64 | Helped |
| add physical loss (ablation) | PhyRecon with rendering uncertainty, physical uncertainty, and physics-guided sampling · ScanNet++ | Object stability SR (%): 60.91 -> 78.16 | Helped |
| add physical loss alone without uncertainty modeling or physics-guided sampling (ablation) | PhyRecon · ScanNet++ | Object reconstruction CD: 4.13 -> 3.97 | Hurt |
| add PhyRecon's physical simulation and joint uncertainty modeling | Object reconstruction · ScanNet++ | Object stability SR (%): 26.43 -> 78.16 | Helped |
| add PhyRecon's physical simulation and joint uncertainty modeling | Object reconstruction · ScanNet | Object stability SR (%): 29.68 -> 70.31 | Helped |
| add PhyRecon's physical simulation and joint uncertainty modeling | Object reconstruction · Replica | Object stability SR (%): 32.89 -> 77.63 | Helped |
Surprising findings
- Adding physical loss without uncertainty and physics-guided sampling yields degenerated object shapes, since the loss overshadows rendering cues under unstable geometry.
- Applying the physical loss to all surface points rather than only contact points contradicts rendering cues and degrades results.
Limitations
- Objects in weakly observed regions may develop degenerate bulges to satisfy physical stability.
- Objects can be reconstructed as disconnected parts that still appear stable in simulation.
- Physical simulation is restricted to rigid bodies modeled as particles and depends on a supporting plane for contact.
Experimental setup
Datasets: ScanNet ScanNet++ Replica
Benchmarks: ScanNet ScanNet++ Replica
Metrics: Chamfer distance F-score normal consistency success rate (stability)
Baselines: MonoSDF RICO ObjectSDF++
Builds on: NeuS MonoSDF ObjectSDF++ RICO DiffTaichi
Closest papers by meaning
- PhysGaia: A Physics-aware Benchmark with Multi-Body Interactions for Dynamic Novel View Synthesis (CVPR 2026 · similarity 0.87)
- GURecon: Learning Detailed 3D Geometric Uncertainties for Neural Surface Reconstruction (AAAI 2025 · similarity 0.87)
- PhyScene3D: Physically Consistent 3D Interactive Tabletop Scene Generation (ICML 2026 · similarity 0.87)
- PBR-NeRF: Inverse Rendering with Physics-Based Neural Fields (CVPR 2025 · similarity 0.86)
- PhysX-3D: Physical-Grounded 3D Asset Generation (NeurIPS 2025 · similarity 0.86)
- Physics-guided Shape-from-Template: Monocular Video Perception through Neural Surrogate Models (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.