Research map / Image generation / Image restoration
Diffusion Reflectance Map: Single-Image Stochastic Inverse Rendering of Illumination and Reflectance
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces DRMNet, a diffusion-based stochastic inverse rendering method that jointly recovers illumination (as a mirror-sphere reflectance map) and object reflectance from a single image of a known-geometry object.
Problem
Single-image inverse rendering to disentangle illumination and reflectance for an object of known geometry is ill-posed because reflectance low-pass filters illumination, so the frequency spectrum of the illumination is lost.
Approach
Formulates radiometric disentanglement as a stochastic forward process on the geometry-invariant reflectance map (reflectance attenuation plus Gaussian noise) and learns its reverse with a diffusion model (IllNet) that steps reflectance toward a perfect mirror, while RefNet jointly estimates the Disney BSDF parameters; an ObsNet diffusion model completes the sparse observed reflectance map.
Key results
- On the iBRDF synthetic dataset, DRMNet achieves illumination logRMSE 2.62 and reflectance logRMSE 0.66, outperforming iBRDF (1.01) and DPI (2.03) on reflectance.
- On nLMVS-Real, DRMNet reaches illumination logRMSE 1.18 in 14.3 s, comparable to ALP (0.95, 267 s) which requires pre-acquired reflectance, and is far faster than iBRDF (5048 s).
- On nLMVS-Real object replacement, DRMNet achieves the best LPIPS 0.17 and PSNR 23.3 among compared methods.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DRMNet instead of iBRDF | single-image inverse rendering · iBRDF synthetic | illumination logRMSE: 3.20 -> 2.62 | Helped |
| Use DRMNet instead of iBRDF | single-image inverse rendering · iBRDF synthetic | reflectance logRMSE: 1.01 -> 0.66 | Helped |
| Use DRMNet instead of ALP with pre-acquired reflectance | illumination estimation · nLMVS-Real | illumination logRMSE: 0.95 -> 1.18 | Hurt |
| Use DRMNet instead of ALP with pre-acquired reflectance | illumination estimation · nLMVS-Real | estimation time (secs): 267 -> 14.3 | Helped |
| Use DRMNet instead of iBRDF | illumination estimation · DeLight-Net | illumination logRMSE: 0.864 -> 0.694 | Helped |
| Use DRMNet instead of iBRDF | relighting · DeLight-Net | relighting logRMSE: 1.027 -> 0.414 | Helped |
| Use DRMNet instead of iBRDF | relighting · nLMVS-Real | relighting logRMSE: 1.69 -> 1.68 | No clear effect |
| Use DRMNet instead of iBRDF | object replacement · nLMVS-Real | object replacement logRMSE: 0.58 -> 0.35 | Helped |
Surprising findings
- The concurrent diffusion-prior method DPI produces illumination estimates that deviate significantly from ground truth on the iBRDF dataset.
Limitations
- Assumes homogeneous reflectance; spatially varying BRDFs are left to future work.
- Reflectance is modeled with a partial Disney BSDF parameterization (diffuse, specular, roughness, metallic), and on DeLight-Net it does not beat all metrics.
- Requires known object geometry (surface normals).
Experimental setup
Datasets: Laval Indoor Dataset Poly Haven HDRIs iBRDF synthetic dataset nLMVS-Real dataset DeLight-Net dataset
Benchmarks: iBRDF synthetic dataset nLMVS-Real dataset DeLight-Net dataset
Metrics: logRMSE PSNR SSIM LPIPS DSSIM relighting error object replacement error estimation time
Baselines: LIME iBRDF DPI ALP DeLight-Net (Georgoulis et al.)
Builds on: denoising diffusion probabilistic models Disney principled BSDF heat dissipation diffusion (Rissanen et al.) iBRDF ALP
Closest papers by meaning
- Diffusion Renderer: Neural Inverse and Forward Rendering with Video Diffusion Models (CVPR 2025 · similarity 0.88)
- Factored-NeuS: Reconstructing Surfaces, Illumination, and Materials of Possibly Glossy Objects (CVPR 2025 · similarity 0.87)
- Channel-wise Noise Scheduled Diffusion for Inverse Rendering in Indoor Scenes (CVPR 2025 · similarity 0.87)
- S^3-Face: SSS-Compliant Facial Reflectance Estimation via Diffusion Priors (CVPR 2025 · similarity 0.87)
- Uni-Renderer: Unifying Rendering and Inverse Rendering Via Dual Stream Diffusion (CVPR 2025 · similarity 0.87)
- DiffusionLight: Light Probes for Free by Painting a Chrome Ball (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.