Research map / 3D vision / Face and shape
Inverse Rendering using Multi-Bounce Path Tracing and Reservoir Sampling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MIRReS, a two-stage mesh-based inverse rendering framework that refines an explicit triangle mesh with physically-based multi-bounce path tracing and reservoir-sampled direct lighting to decompose geometry, materials, and lighting from multi-view images.
Problem
Inverse rendering of geometry, materials and lighting from multi-view images is ill-posed; implicit fields lack physical constraints for indirect lighting, and prior mesh-based methods are unstable and noisy under Monte Carlo path tracing.
Approach
Initialize with a NeRF/NeuS2 coarse mesh, then refine per-vertex offsets on a fixed-topology mesh and jointly optimize PBR materials and HDR environment light with three-bounce path tracing for indirect light and RIS/reservoir sampling (spatial and temporal reuse) for direct light.
Key results
- On the TensoIR dataset, MIRReS reports relighting PSNR 32.363 vs. 28.580 for TensoIR (second best), and albedo PSNR 32.348.
- On the OWL dataset, relighting PSNR 28.827 vs. 26.382 for TensoIR.
- Ablation: reservoir sampling plus multi-bounce raise relighting PSNR from 28.992 (neither) to 33.788 (both).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add reservoir sampling without multi-bounce ray tracing (ablation) | MIRReS · TensoIR dataset | Albedo PSNR: 31.950 -> 32.529 | Helped |
| add reservoir sampling without multi-bounce ray tracing (ablation) | MIRReS · TensoIR dataset | Relighting PSNR: 28.992 -> 31.239 | Helped |
| add multi-bounce ray tracing without reservoir sampling (ablation) | MIRReS · TensoIR dataset | Albedo PSNR: 31.950 -> 33.752 | Helped |
| add multi-bounce ray tracing without reservoir sampling (ablation) | MIRReS · TensoIR dataset | Relighting PSNR: 28.992 -> 31.694 | Helped |
| add reservoir sampling to multi-bounce ray tracing (ablation) | MIRReS · TensoIR dataset | Albedo PSNR: 33.752 -> 34.348 | Helped |
| add reservoir sampling to multi-bounce ray tracing (ablation) | MIRReS · TensoIR dataset | Relighting PSNR: 31.694 -> 33.788 | Helped |
| add multi-bounce ray tracing to reservoir sampling (ablation) | MIRReS · TensoIR dataset | Albedo PSNR: 32.529 -> 34.348 | Helped |
| add multi-bounce ray tracing to reservoir sampling (ablation) | MIRReS · TensoIR dataset | Relighting PSNR: 31.239 -> 33.788 | Helped |
| use MIRReS instead of the strongest reported baseline | Inverse rendering method · TensoIR dataset | Albedo PSNR: 30.286 -> 32.348 | Helped |
| use MIRReS instead of the strongest reported baseline | Inverse rendering method · OWL (4 selected scenes) | Relighting PSNR: 26.382 -> 28.827 | Helped |
| use more than 32 SPP (ablation) | MIRReS · Hotdog scene | Novel view synthesis PSNR | No clear effect |
| use surface rendering alone, without neural radiance field rendering (ablation) | MIRReS · Geometry refinement and material reconstruction | geometry accuracy | Hurt |
Surprising findings
- Path tracing without indirect illumination ('PT') can have higher novel-view PSNR than with indirect illumination at low SPP, because indirect illumination at low sample counts misses specular reflections, which the authors explain as a noise effect.
Limitations
- Final geometry quality depends on the stage-1 coarse mesh from NeuS2, which has noticeable errors in specular or fine-detail regions.
- Gradients of non-primary rays are not propagated (indirect rays are detached) due to GPU memory limits.
Experimental setup
Datasets: TensoIR synthetic dataset Objects-with-Lighting (OWL) dataset
Benchmarks: TensoIR synthetic dataset Objects-with-Lighting dataset
Metrics: CD normal MAE PSNR SSIM LPIPS
Baselines: TensoIR GS-IR NVdiffrec-MC NeRFactor NeILF++
Builds on: NeRF2Mesh NVdiffrec-MC ReSTIR Instant-NGP NeuS2
Closest papers by meaning
- Multi-times Monte Carlo Rendering for Inter-reflection Reconstruction (NeurIPS 2024 · similarity 0.91)
- GI-GS: Global Illumination Decomposition on Gaussian Splatting for Inverse Rendering (ICLR 2025 · similarity 0.90)
- SVG-IR: Spatially-Varying Gaussian Splatting for Inverse Rendering (CVPR 2025 · similarity 0.88)
- Inverse Rendering of Glossy Objects via the Neural Plenoptic Function and Radiance Fields (CVPR 2024 · similarity 0.88)
- Eclipse: Disambiguating Illumination and Materials using Unintended Shadows (CVPR 2024 · similarity 0.88)
- Differentiable Point-based Inverse Rendering (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.