Research map / 3D vision / Gaussian splatting
GS-IR: 3D Gaussian Splatting for Inverse Rendering
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GS-IR, extending 3D Gaussian Splatting to inverse rendering (geometry, PBR material, and environment lighting) from multi-view images under unknown lighting, using depth-derivation normal regularization and SH-baked occlusion for indirect light.
Problem
Inverse rendering of geometry, material, and illumination from multi-view images under unknown lighting is limited in NeRF-based methods by low expressiveness and high compute; 3DGS lacks native normals and forward splatting cannot trace occlusion for indirect lighting.
Approach
Derive pseudo normals from the depth gradient of a linearly interpolated Gaussian depth, supervise per-Gaussian normals with it, and bake occlusion from six depth cubemap passes into spherical harmonics volumes to model indirect illumination within a 3DGS split-sum PBR pipeline.
Key results
- On TensoIR Synthetic, GS-IR reaches 35.333 PSNR novel view synthesis, above TensoIR's 35.088, and albedo PSNR 30.286 vs 29.275 for TensoIR.
- GS-IR's normal MAE is 4.948, slightly worse than TensoIR's 4.100, and its relighting PSNR of 24.374 ranks second behind TensoIR's 28.580.
- On Mip-NeRF 360, GS-IR achieves 25.381 PSNR and trains in about 45 minutes, versus 27.21 PSNR and about 35 minutes for 3DGS.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use linear interpolation for depth and optimize Gaussian-stored normals from depth-derived normals (ablation) | GS-IR · TensoIR Synthetic | Normal MAE: 6.218 -> 4.948 | Helped |
| Use linear interpolation for depth and optimize Gaussian-stored normals from depth-derived normals (ablation) | GS-IR · Mip-NeRF 360 | PSNR: 23.119 -> 25.381 | Helped |
| Use volumetric accumulation for depth instead of linear interpolation (ablation) | GS-IR · TensoIR Synthetic | Normal MAE: 4.948 -> 16.347 | Hurt |
| Use peak selection for depth instead of linear interpolation (ablation) | GS-IR · TensoIR Synthetic | Normal MAE: 4.948 -> 9.466 | Hurt |
| Use volumetric accumulation for depth instead of linear interpolation (ablation) | GS-IR · TensoIR Synthetic | Normal MAE: 4.948 -> 9.315 | Hurt |
| Use peak selection for depth instead of linear interpolation (ablation) | GS-IR · TensoIR Synthetic | Normal MAE: 4.948 -> 7.986 | Hurt |
| Remove occlusion volume (ablation) | GS-IR · TensoIR Synthetic | PSNR: 34.997 -> 35.333 | Hurt |
| Remove occlusion volume (ablation) | GS-IR · Mip-NeRF 360 | PSNR: 25.060 -> 25.381 | Hurt |
| Remove indirect illumination (ablation) | GS-IR · TensoIR Synthetic | PSNR: 35.186 -> 35.333 | Hurt |
| Remove indirect illumination (ablation) | GS-IR · Mip-NeRF 360 | PSNR: 24.898 -> 25.381 | Hurt |
Surprising findings
- Normal estimation on TensoIR Synthetic is slightly worse than TensoIR (MAE 4.948 vs 4.100), despite better novel view synthesis and albedo.
- Ablation shows linear depth interpolation beats both volumetric accumulation and peak selection for normals.
Limitations
- Spherical harmonics only represent low-frequency occlusion and indirect light, so only the diffuse indirect term is modeled.
- Specular indirect illumination is not modeled.
- Normal estimation is slightly inferior to TensoIR on the synthetic benchmark.
Experimental setup
Datasets: TensoIR Synthetic Mip-NeRF 360
Benchmarks: TensoIR Synthetic Mip-NeRF 360
Metrics: MAE PSNR SSIM LPIPS runtime
Baselines: NeRFactor InvRender NVDiffrec TensoIR NeRF++ Plenoxels Instant-NGP Mip-NeRF 360 3DGS
Builds on: 3D Gaussian Splatting TensoRF split-sum approximation Cook-Torrance microfacet model
Closest papers by meaning
- GI-GS: Global Illumination Decomposition on Gaussian Splatting for Inverse Rendering (ICLR 2025 · similarity 0.94)
- SVG-IR: Spatially-Varying Gaussian Splatting for Inverse Rendering (CVPR 2025 · similarity 0.91)
- Speedy-Splat: Fast 3D Gaussian Splatting with Sparse Pixels and Sparse Primitives (CVPR 2025 · similarity 0.91)
- Generative Sparse-View Gaussian Splatting (CVPR 2025 · similarity 0.91)
- MS-GS: Multi-Appearance Sparse-View 3D Gaussian Splatting in the Wild (NeurIPS 2025 · similarity 0.91)
- PUP 3D-GS: Principled Uncertainty Pruning for 3D Gaussian Splatting (CVPR 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.