Research map / 3D vision / Face and shape
Differentiable Inverse Rendering with Interpretable Basis BRDFs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a differentiable inverse rendering method that jointly optimizes 2D Gaussians and basis BRDFs with adaptive basis count (merge/removal) and sparsity regularization to obtain interpretable, spatially separated SVBRDFs from multi-view flash photography.
Problem
Existing basis-BRDF inverse rendering methods produce spatially entangled, non-interpretable basis BRDFs with a fixed number of bases, limiting scene editing and generalization across scene complexity.
Approach
Model geometry as 2D Gaussians each blending basis BRDFs, dynamically merge/remove basis BRDFs during optimization, and impose entropy-based sparsity on per-Gaussian and per-pixel weights, plus a specular-weighted rendering loss.
Key results
- Normal reconstruction MAE of 9.81 versus 15.28 (GS3), 15.71 (IRON), and 17.04 (DPIR) on the synthetic dataset.
- Training time of 0.5 h, compared with 1 h (GS3), 2 h (DPIR), and 12 h (IRON).
- Produces interpretable, spatially separated basis BRDFs whose number adapts to scene complexity, enabling reflectance editing and relighting.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the full method with 2D Gaussians and interpretable basis BRDFs instead of IRON, DPIR, or GS3 | Inverse rendering on the synthetic dataset · Synthetic dataset | normal reconstruction MAE: 15.71 -> 9.81; 17.04 -> 9.81; 15.28 -> 9.81 | Helped |
| Use the full method instead of IRON, DPIR, or GS3 | Inverse rendering on the synthetic dataset · Synthetic dataset | training time: 12h -> 0.5h; 2h -> 0.5h; 1h -> 0.5h | Helped |
| Add sparsity regularization and basis BRDF merge and removal control (ablation) | Ours (Section 3.1) · Synthetic scenes | basis BRDF interpretability | Helped |
| Add specular-weighted rendering loss using the rendered half-way angle map (ablation) | Ours · Synthetic scenes | geometry and reflectance reconstruction quality | Helped |
Surprising findings
- GS3 yields good relighting despite inaccurate normals, because its neural network compensates via residual learning.
Limitations
- Basis BRDF representation assumes opaque objects; non-surface objects not modeled.
- Does not yet incorporate semantic information from large vision models.
Experimental setup
Datasets: synthetic multi-object flash photography scenes (4 scenes, rendered) real-world mobile phone flash capture (1 scene)
Benchmarks: synthetic multi-object flash photography dataset real-world multi-object scene
Metrics: mean angular error (MAE) of normals training time
Baselines: IRON DPIR GS3
Builds on: 2D Gaussian splatting Disney BRDF model basis BRDF representation GS3
Closest papers by meaning
- Differentiable Point-based Inverse Rendering (CVPR 2024 · similarity 0.89)
- SVG-IR: Spatially-Varying Gaussian Splatting for Inverse Rendering (CVPR 2025 · similarity 0.87)
- RobIR: Robust Inverse Rendering for High-Illumination Scenes (NeurIPS 2024 · similarity 0.86)
- PBR-NeRF: Inverse Rendering with Physics-Based Neural Fields (CVPR 2025 · similarity 0.86)
- Factored-NeuS: Reconstructing Surfaces, Illumination, and Materials of Possibly Glossy Objects (CVPR 2025 · similarity 0.85)
- Inverse Rendering using Multi-Bounce Path Tracing and Reservoir Sampling (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.