Research map / 3D vision / NeRF and radiance fields
Neural Inverse Rendering from Propagating Light
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a time-resolved neural radiance cache with physically-based inverse rendering to reconstruct geometry and render propagating light from multi-view flash lidar videos, including direct/indirect decomposition and relighting.
Problem
Inverting multi-viewpoint, time-resolved flash lidar measurements of propagating light while accounting for indirect (multi-bounce) light transport, which prior lidar neural methods ignore or model without physical grounding.
Approach
Extends neural radiance caching to the time domain: a NeRF-based cache stores infinite-bounce time-resolved radiance at any point and direction, split into direct and indirect components with a split-sum approximation, and is optimized inside a physically-based time-resolved renderer with Disney-GGX BRDF.
Key results
- On simulated scenes, the method reaches 30.99 dB PSNR and 8.45 normal MAE, versus 29.00 dB and 22.80 for FWP++.
- On captured scenes, FWP++ slightly outperforms the method on view synthesis (28.45 dB vs 27.39 dB PSNR), while the method reports comparable transient IOU (0.54 vs 0.55).
- The method demonstrates time-resolved relighting and training from CW-ToF or steady-state images, but requires more than one day of optimization on a single GPU.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a physically based time-resolved radiance cache instead of FWP++'s less constrained model | Ours versus FWP++ · sim | L1 depth: 0.47 -> 0.21 | Helped |
| Use a physically based time-resolved radiance cache instead of FWP++'s less constrained model | Ours versus FWP++ · sim | PSNR (dB): 29.00 -> 30.99 | Helped |
| Use a physically based time-resolved radiance cache instead of FWP++'s less constrained model | Ours versus FWP++ · sim | LPIPS: 0.30 -> 0.31 | Hurt |
| Use a physically based time-resolved radiance cache instead of FWP++'s less constrained model | Ours versus FWP++ · cap | PSNR (dB): 28.45 -> 27.39 | Hurt |
| Use a physically based time-resolved radiance cache instead of FWP++'s less constrained model | Ours versus FWP++ · cap | T-IOU: 0.55 -> 0.54 | Hurt |
Surprising findings
- On captured data, the physically-based method slightly underperforms the less constrained FWP++ baseline on view synthesis, which the authors attribute to calibration sensitivity and model mismatch.
Limitations
- More constrained physical model leads to degraded performance versus baselines on captured data due to model mismatch.
- Training takes more than one day on a single GPU due to physical light transport simulation and large time-resolved vectors.
Experimental setup
Datasets: Mitsuba 2 simulated multi-view transient data (Cornell box, pots, peppers, kitchen) captured multi-view flash lidar data (globe, house, spheres, statue)
Benchmarks: simulated multi-view transient scenes captured flash lidar scenes
Metrics: PSNR SSIM LPIPS MAE L1 depth transient IOU
Baselines: T-NeRF FWP++
Builds on: Flying with Photons T-NeRF Zip-NeRF radiance caching NeRF
Closest papers by meaning
- Inverse Rendering of Glossy Objects via the Neural Plenoptic Function and Radiance Fields (CVPR 2024 · similarity 0.88)
- VMINer: Versatile Multi-view Inverse Rendering with Near- and Far-field Light Sources (CVPR 2024 · similarity 0.88)
- UniRelight: Learning Joint Decomposition and Synthesis for Video Relighting (NeurIPS 2025 · similarity 0.88)
- Diffusion Renderer: Neural Inverse and Forward Rendering with Video Diffusion Models (CVPR 2025 · similarity 0.87)
- IllumiNeRF: 3D Relighting Without Inverse Rendering (NeurIPS 2024 · similarity 0.87)
- RobIR: Robust Inverse Rendering for High-Illumination Scenes (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.