Research map / 3D vision / NeRF and radiance fields
SplitNeRF: Split Sum Approximation Neural Field for Joint Geometry, Illumination, and Material Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SplitNeRF, a NeRF-based inverse rendering method that models pre-integrated split-sum image-based lighting with a single MLP plus Monte Carlo regularization and supervises self-occlusion factors, recovering geometry, materials and lighting in about 1 hour on one A100.
Problem
Digitizing real objects by jointly estimating geometry, material properties and environment lighting from posed images under fixed lighting, while keeping training efficient and recovering high-frequency illumination and self-occlusion.
Approach
Represent the split-sum pre-integrated environment lighting as an MLP over direction and roughness (instead of mipmaps), train it with a Monte Carlo-based regularizer that matches roughness-blurred lighting, and supervise a separate occlusion MLP with Monte Carlo visibility estimates computed on the predicted geometry.
Key results
- Outperforms all baselines on NeRFactor relighting metrics for all but one scene, and on Blender relighting across all metrics per the paper's claim.
- Achieves state-of-the-art relighting with about 47 minutes of training on a single A100 GPU (about 1 hour).
- Mipmap illumination variant gives slight gains in relighting and albedo but increases training time by about 36% (47 to 64 min).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use SplitNeRF instead of the evaluated baselines | NeRFactor, Blender, and Shiny Blender · NeRFactor, Blender, and Shiny Blender | relighting quality | Helped |
| add the supervised occlusion loss (ablation) | SplitNeRF · NeRFactor | relighting PSNR | Helped |
| add the supervised occlusion loss (ablation) | SplitNeRF · NeRFactor | albedo reconstruction | Helped |
| penalize metalness predictions with material regularization (ablation) | SplitNeRF · NeRFactor | quality | Helped |
| supervise occlusion factors with their per-channel averages (ablation) | SplitNeRF · NeRFactor | relighting quality | Helped |
| supervise occlusion factors with their per-channel averages (ablation) | SplitNeRF · NeRFactor | albedo reconstruction | Helped |
| replace the regularized illumination MLP with a mipmap representation (ablation) | SplitNeRF · NeRFactor | relighting quality | Helped |
| replace the regularized illumination MLP with a mipmap representation (ablation) | SplitNeRF · NeRFactor | albedo reconstruction | Helped |
| replace the regularized illumination MLP with a mipmap representation (ablation) | SplitNeRF · NeRFactor | training time: 47 min. -> 64 min.; 36% increase | Hurt |
Surprising findings
- Supervising occlusion with per-channel averages instead of full per-channel factors improves both relighting and albedo, suggesting per-channel occlusion is noisy.
Limitations
- Image-based lighting assumes lights at infinite distance, causing errors when violated.
- Ignores indirect illumination and thus inter-reflections.
- Cannot model transmission or subsurface scattering.
Experimental setup
Datasets: NeRFactor dataset NeRF Blender dataset (extended with high-frequency lighting) Shiny Blender (RefNeRF) dataset (extended with high-frequency lighting)
Benchmarks: NeRFactor relighting dataset Blender relighting Shiny Blender relighting NeRFactor albedo
Metrics: PSNR SSIM LPIPS
Baselines: NeRFactor NVDiffRec NVDiffRecMC NMF NeRO
Builds on: NeuS NeRF Instant-NGP hash grid encoding split sum approximation
Closest papers by meaning
- A Pre-convolved Representation for Plug-and-Play Neural Illumination Fields (AAAI 2024 · similarity 0.88)
- IllumiNeRF: 3D Relighting Without Inverse Rendering (NeurIPS 2024 · similarity 0.88)
- Neural LightRig: Unlocking Accurate Object Normal and Material Estimation with Multi-Light Diffusion (CVPR 2025 · similarity 0.87)
- RNG: Relightable Neural Gaussians (CVPR 2025 · similarity 0.87)
- UniRelight: Learning Joint Decomposition and Synthesis for Video Relighting (NeurIPS 2025 · similarity 0.86)
- Few-Shot Neural Radiance Fields under Unconstrained Illumination (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.