Research map / 3D vision / Face and shape
Sensing Surface Patches in Volume Rendering for Inferring Signed Distance Functions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Infers SDFs from multi-view images via volume rendering by sensing surface patches (pulling Gaussian-sampled points onto the zero level set) and imposing depth, photometric (NCC) and surface-fitting constraints on them.
Problem
Neural SDF reconstruction from multi-view RGB via volume rendering only constrains single ray-surface intersection points, leading to surface drift, artifacts in empty space, and missing geometric detail.
Approach
Use the SDF value and gradient to pull randomly sampled points near a ray-surface intersection (or back-projected depth anchor) onto the zero level set, forming a surface patch on which explicit surface constraints (depth consistency, NCC photo-consistency, surface fitting) are imposed alongside volume rendering losses.
Key results
- On ScanNet (MonoSDF setup), the method reaches F-score 0.797 and CD-L1 0.037 vs. MonoSDF's 0.733 and 0.042.
- On NeuralRGBD ScanNet subsets, mean F-score 83.79 vs. NeuralRGBD's 66.90.
- On Replica, mean accuracy 2.33 cm and normal consistency 93.31 vs. NICER-SLAM's 3.65 cm and 90.27.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use surface patch depth consistency L_DC (ablation) | Ours-Full · ScanNet (MonoSDF setup) | F-score: 0.733 -> 0.766 | Helped |
| use surface patch photometric consistency L_NCC (ablation) | Ours-Full · ScanNet (MonoSDF setup) | F-score: 0.733 -> 0.772 | Helped |
| use surface fitting loss L_Fit (ablation) | Ours-Full · ScanNet (MonoSDF setup) | F-score: 0.733 -> 0.785 | Helped |
| increase all three surface constraint loss weights by 10× (ablation) | Ours-Full · ScanNet (MonoSDF setup) | F-score: 0.797 -> 0.759 | Hurt |
| reduce all three surface constraint loss weights to 0.1× (ablation) | Ours-Full · ScanNet (MonoSDF setup) | F-score: 0.797 -> 0.743 | Hurt |
| add mask w_j to rule out outliers during pulling (ablation) | Ours-Full · ScanNet (MonoSDF setup) | F-score: 0.770 -> 0.797 | Helped |
| add confidence weight eta_j for patch points (ablation) | Ours-Full · ScanNet (MonoSDF setup) | F-score: 0.784 -> 0.797 | Helped |
| pull sampled points onto the zero level set before applying surface constraints (ablation) | Ours-Full · ScanNet (MonoSDF setup) | F-score: 0.753 -> 0.797 | Helped |
| increase patch samples from 1 to 9 (ablation) | Ours-Full · ScanNet (MonoSDF setup) | F-score: 0.760 -> 0.797 | Helped |
| increase patch samples from 9 to 16 (ablation) | Ours-Full · ScanNet (MonoSDF setup) | F-score: 0.797 -> 0.777 | Hurt |
| replace patch-level photo consistency with pixel-level photo consistency (ablation) | Ours-Full · ScanNet (MonoSDF setup) | F-score: 0.797 -> 0.794 | Hurt |
| use half the baseline Gaussian variance (ablation) | Ours-Full · ScanNet (MonoSDF setup) | F-score: 0.797 -> 0.763 | Hurt |
Surprising findings
- Pixel-level photometric consistency is worse than patch-level photometric consistency in the ablation (F-score 0.794 vs 0.797 on the ScanNet MonoSDF split, with recall better for pixels), and Neuralangelo performs poorly on real indoor ScanNet scenes despite tuning.
Limitations
- Too large a variance tau^2 covers areas where points are ruled out by masks, reducing efficiency and performance.
- Relies on monocular or sensor depth and normal cues as supervision, with scale/shift alignment of Omnidata depth needed.
- Gains on the ScanNet MonoSDF split are small relative to MonoSDF for accuracy (0.036 vs 0.035).
Experimental setup
Datasets: ScanNet Replica Tanks and Temples NeuralRGBD ScanNet subsets
Benchmarks: ScanNet Replica NeuralRGBD ScanNet subsets Tanks and Temples
Metrics: Accuracy Completeness Chamfer distance (CD-L1) Precision Recall F-score Normal consistency
Baselines: COLMAP UNISURF NeuS VolSDF Manhattan-SDF NeuRIS Neuralangelo MonoSDF NeuralRGBD NICE-SLAM DROID-SLAM NICER-SLAM
Builds on: VolSDF NeuS MonoSDF NeRF
Closest papers by meaning
- Multi-View Reconstruction Using Signed Ray Distance Functions (SRDF) (CVPR 2023 · similarity 0.91)
- Neural Signed Distance Function Inference through Splatting 3D Gaussians Pulled on Zero-Level Set (NeurIPS 2024 · similarity 0.89)
- S2MDF: A Plug-And-Play Layer for Intersection-Free Multi-Object Signed Distance Fields (NeurIPS 2026 · similarity 0.88)
- Learning Bijective Surface Parameterization for Inferring Signed Distance Functions from Sparse Point Clouds with Grid Deformation (CVPR 2025 · similarity 0.87)
- DUDF: Differentiable Unsigned Distance Fields with Hyperbolic Scaling (CVPR 2024 · similarity 0.86)
- Geometry Field Splatting with Gaussian Surfels (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.