Research map / Image generation / Autoregressive and latent generation
Intrinsic Image Diffusion for Indoor Single-view Material Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Fine-tunes Stable Diffusion V2 as a conditional latent diffusion model that samples multiple plausible albedo, roughness, and metallic maps from a single indoor image, trained on synthetic InteriorVerse data.
Problem
Single-view indoor material (intrinsic appearance) estimation is highly ambiguous between lighting and material, and real labeled datasets are lacking, so deterministic models average over plausible solutions and produce over-smoothed, baked-in results.
Approach
Reformulate appearance decomposition probabilistically by sampling material maps from a conditional diffusion model, and adapt the real-image prior of pretrained Stable Diffusion to material estimation so that it generalizes from synthetic to real images.
Key results
- Albedo FID on InteriorVerse improves from 46.21 (Zhu et al.) to 25.42, a 44.99% relative gain, with mean-sample PSNR 17.42 vs 15.92.
- Roughness and metallic FID improve over Zhu et al. by 39.90% and 53.56%, respectively.
- Image reconstruction on real images reaches 21.96 PSNR with the full lighting pipeline, versus 13.54 for Zhu et al.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use probabilistic material sampling instead of deterministic prediction | Intrinsic Image Diffusion · InteriorVerse albedo | FID: 46.21 -> 25.42 | Helped |
| Use probabilistic material sampling instead of deterministic prediction | Intrinsic Image Diffusion · InteriorVerse roughness | FID: 69.92 -> 42.02 | Helped |
| Use probabilistic material sampling instead of deterministic prediction | Intrinsic Image Diffusion · InteriorVerse metallic | FID: 48.71 -> 22.62 | Helped |
| Use our material predictions in Zhu et al.'s inverse-rendering pipeline | Zhu et al. [48] · Real-world image reconstruction | PSNR: 13.54 ± 2.22 -> 14.07 ± 1.91 | Helped |
| Add our lighting optimization to our material predictions | Intrinsic Image Diffusion · Real-world image reconstruction | PSNR: 14.07 ± 1.91 -> 21.96± 3.22 | Helped |
| Add Stable Diffusion V2 pre-training before fine-tuning on material estimation (ablation) | Intrinsic Image Diffusion · InteriorVerse | FID: 113.31 -> 25.42 | Helped |
| Add Stable Diffusion V2 pre-training before fine-tuning on material estimation (ablation) | Intrinsic Image Diffusion · InteriorVerse | PSNR: 13.38 ± 2.44 -> 17.42 ± 3.08 | Helped |
| Add Stable Diffusion V2 pre-training before fine-tuning on material estimation (ablation) | Intrinsic Image Diffusion · IIW | WHDR: 35.60 ± 12.94 -> 22.02 ± 11.99 | Helped |
| Replace diffusion with GAN-based pix2pix (ablation) | Material estimation · InteriorVerse | FID: 25.42 -> 84.28 | Hurt |
Surprising findings
- On real IIW, WHDR scores are similar to baselines, and the paper argues WHDR favors over-smoothed outputs and fails to reflect high-frequency quality.
Limitations
- Training requires paired synthetic data; extension to weak or real supervision is left to future work.
- Lighting is optimized independently from material estimation rather than jointly.
Experimental setup
Datasets: InteriorVerse IIW ScanNet++
Benchmarks: InteriorVerse IIW ScanNet++
Metrics: PSNR SSIM LPIPS FID WHDR user preference
Baselines: Li et al. (Li et al. 2020 spatially-varying inverse rendering) Zhu et al. (InteriorVerse) IIW optimization method (Bell et al.) Pix2Pix
Builds on: Stable Diffusion Zhu et al. InteriorVerse inverse rendering Latent diffusion models DDIM
Closest papers by meaning
- IDArb: Intrinsic Decomposition for Arbitrary Number of Input Views and Illuminations (ICLR 2025 · similarity 0.90)
- Alchemist: Parametric Control of Material Properties with Diffusion Models (CVPR 2024 · similarity 0.89)
- Material Palette: Extraction of Materials from a Single Image (CVPR 2024 · similarity 0.89)
- Material Anything: Generating Materials for Any 3D Object via Diffusion (CVPR 2025 · similarity 0.88)
- Channel-wise Noise Scheduled Diffusion for Inverse Rendering in Indoor Scenes (CVPR 2025 · similarity 0.88)
- Neural LightRig: Unlocking Accurate Object Normal and Material Estimation with Multi-Light Diffusion (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.