Research map / 3D vision / Face and shape
Differentiable Point-based Inverse Rendering
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DPIR, a point-based differentiable inverse rendering method that estimates shape and spatially-varying BRDF from multi-view multi-light or photometric images using splatting plus a hybrid SDF-point geometry and regularized basis BRDFs.
Problem
Inverse rendering of geometry and spatially-varying BRDF from images under diverse illumination is slow in volumetric (NeRF-style) formulations, which need multiple samples per ray, and ill-posed due to limited light-view angular samples.
Approach
Uses point-based splatting as the forward renderer in analysis-by-synthesis inverse rendering, combining SDF-derived normals on movable points with a regularized (positive, l1-bounded) specular basis-BRDF representation and a point-based shadow map for visibility.
Key results
- On DiLiGenT-MV, DPIR achieves normal MAE 4.35 on Bear vs 4.68 for PS-NeRF, with about 10x faster training (2h vs 22h) and 8x lower memory (5MB vs 40MB) than PS-NeRF.
- On the photometric dataset, DPIR reaches PSNR 35.56 vs 31.91 for IRON, with 6x faster training (2h vs 12h) and 5x lower memory (5MB vs 28MB).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DPIR instead of IRON | Photometric inverse rendering · Photometric dataset | PSNR: 31.91 -> 35.56 | Helped |
| Use DPIR instead of IRON | Photometric inverse rendering · Photometric dataset | training time: 12h -> 2h | Helped |
| Use DPIR instead of PS-NeRF | DiLiGenT-MV inverse rendering · DiLiGenT-MV Bear, single light | PSNR: 35.19 -> 43.21 | Helped |
| Add point-based visibility testing (ablation) | DPIR · Occluded regions | surface normal accuracy | Helped |
| Remove the SDF from the hybrid shape representation (ablation) | DPIR · Geometry and appearance reconstruction | reconstruction quality | Hurt |
| Add regularization to specular-basis coefficients (ablation) | DPIR · Diffuse/specular appearance decomposition | decomposition accuracy | Helped |
| Optimize point radii dynamically (ablation) | DPIR · Geometry reconstruction | geometry accuracy | Helped |
| Use mask inputs for point initialization and mask loss (ablation) | DPIR · Geometry reconstruction | reconstruction quality | Helped |
Surprising findings
- Mask inputs are not strictly required: DPIR often yields plausible reconstructions without masks.
Limitations
- Models only direct reflection, ignoring global illumination such as inter-reflections.
- Does not explicitly model transmission, limiting applicability to diverse materials.
Experimental setup
Datasets: DiLiGenT-MV photometric dataset rendered from four objects
Benchmarks: DiLiGenT-MV photometric dataset
Metrics: PSNR SSIM LPIPS MAE training time memory footprint
Baselines: PhySG TensoIR PS-NeRF IRON
Builds on: NeuS Point-NeRF PS-NeRF 3D Gaussian Splatting
Closest papers by meaning
- Differentiable Inverse Rendering with Interpretable Basis BRDFs (CVPR 2025 · similarity 0.89)
- Inverse Rendering using Multi-Bounce Path Tracing and Reservoir Sampling (ICLR 2025 · similarity 0.88)
- RobIR: Robust Inverse Rendering for High-Illumination Scenes (NeurIPS 2024 · similarity 0.86)
- Factored-NeuS: Reconstructing Surfaces, Illumination, and Materials of Possibly Glossy Objects (CVPR 2025 · similarity 0.86)
- LinPrim: Linear Primitives for Differentiable Volumetric Rendering (NeurIPS 2025 · similarity 0.85)
- SVG-IR: Spatially-Varying Gaussian Splatting for Inverse Rendering (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.