Research map / 3D vision / NeRF and radiance fields
Light Transport-aware Diffusion Posterior Sampling for Single-View Reconstruction of 3D Volumes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Cloudy, a 1,000-cloud synthetic volume dataset, and a monoplanar latent diffusion prior combined with a differentiable volume path tracer via parametric diffusion posterior sampling (PDPS) for single-view cloud reconstruction.
Problem
Single-view reconstruction of volumetric fields (e.g., clouds) where multiple light scattering and unknown lighting/background make the inverse problem highly ill-posed, and no public 3D volumetric training dataset exists for diffusion priors.
Approach
Trains an unconditional latent diffusion model on a diffusion-friendly monoplanar implicit latent representation of simulated clouds, and couples diffusion posterior sampling with a physically-based differentiable volume path tracer that jointly optimizes physical scene parameters (lighting, material) while steering the latent diffusion denoising process.
Key results
- Monoplanar latent achieves PSNR 38.46 vs 38.13 for triplanar and 37.26 for dense grid on cloud reconstruction.
- DPS with three views (DPS3x3) reaches LPIPS 0.0124 test-view (T-LPIPS) and 0.0572 novel-view (N-LPIPS), outperforming DRT3 (0.0242 / 0.1188) and SPS3 (0.0118 / 0.1081) on novel views.
- Single-view DPS1 reconstructs in about 4 minutes versus roughly 30-40 minutes for DRT and SPS baselines.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use three-view DPS with three noisy restarts instead of SPS3 | DPS · Cloud reconstruction test set | N-LPIPS: 0.1081 -> 0.0572 | Helped |
| use three-view DPS instead of DRT3 | DPS · Cloud reconstruction test set | N-LPIPS: 0.1188 -> 0.0581 | Helped |
| use three-view DPS instead of SPS3 | DPS · Cloud reconstruction test set | Time: 00:33:54 -> 00:04:23 | Helped |
| use three-view DPS instead of SPS3 | DPS · Cloud reconstruction test set | T-LPIPS: 0.0118 -> 0.0241 | Hurt |
| add monoplanar representation rather than using triplanar representation (ablation) | Volume latent representation · Cloudy dataset | PSNR: 38.13 -> 38.46 | Helped |
| add monoplanar representation rather than using dense grid representation (ablation) | Volume latent representation · Cloudy dataset | PSNR: 37.26 -> 38.46 | Helped |
| add monoplanar representation rather than using triplanar representation (ablation) | Volume latent representation · Cloudy dataset | RMSE: 0.01245 -> 0.01199 | Helped |
| add monoplanar representation rather than using dense grid representation (ablation) | Volume latent representation · Cloudy dataset | RMSE: 0.01377 -> 0.01199 | Helped |
| add monoplanar representation rather than using triplanar representation (ablation) | Volume latent representation · Cloudy dataset | SSIM: 0.8547 -> 0.8609 | Helped |
| add monoplanar representation rather than using dense grid representation (ablation) | Volume latent representation · Cloudy dataset | SSIM: 0.8412 -> 0.8609 | Helped |
Surprising findings
- Single-view DPS1 novel-view LPIPS (0.1126) beats multi-view DRT3 (0.1188) despite using only one view
- Single-view DPS1 target-view LPIPS (0.0205) is worse than SPS1 (0.0123), a limitation at the observed view
Limitations
- Ambiguity between diffusion latent theta and physical parameters phi; background radiance may be confused with cloud structure without regularization
- Pre-trained diffusion model requires days to compute latent encoding and reconstruction is computationally intensive due to path tracing
- Evaluated only on cumulus clouds; generalization to smoke, fire, or time-varying phenomena is not shown
Experimental setup
Datasets: Cloudy dataset
Benchmarks: Cloudy dataset test cases
Metrics: PSNR RMSE MAE SSIM LPIPS reconstruction time
Baselines: Differentiable Ratio-Tracking (DRT) Singular Path Sampling (SPS) Triplanar latent representation Dense grid latent representation
Builds on: DPS (diffusion posterior sampling) Singular Path Sampling triplane representation latent diffusion models
Closest papers by meaning
- Adaptive 3D Reconstruction via Diffusion Priors and Forward Curvature-Matching Likelihood Updates (NeurIPS 2025 · similarity 0.87)
- DiffusionLight: Light Probes for Free by Painting a Chrome Ball (CVPR 2024 · similarity 0.87)
- Wonderland: Navigating 3D Scenes from a Single Image (CVPR 2025 · similarity 0.86)
- Diffusion Renderer: Neural Inverse and Forward Rendering with Video Diffusion Models (CVPR 2025 · similarity 0.86)
- MicroDiffusion: Implicit Representation-Guided Diffusion for 3D Reconstruction from Limited 2D Microscopy Projections (CVPR 2024 · similarity 0.86)
- Denoising Diffusion via Image-Based Rendering (ICLR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.