Research map / 3D vision / NeRF and radiance fields
SpecNeRF: Gaussian Directional Encoding for Specular Reflections
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a learnable 3D Gaussian directional encoding for NeRF specular reflections under near-field lighting, plus monocular normal supervision early in training to resolve shape-radiance ambiguity.
Problem
NeRFs struggle to model view-dependent specular reflections on glossy surfaces under spatially varying near-field lighting in room-scale indoor scenes, because existing directional encodings (Fourier, SH, integrated directional encoding) are spatially invariant and assume distant lighting.
Approach
Replace spatially invariant directional encodings with a set of learnable 3D Gaussians whose closed-form ray-maximum projections form the encoding, scaled by roughness to emulate prefiltered environment maps; Gaussians are initialized by training an incident light field on blurred image pyramids, and monocular normals supervise geometry only at the start of training.
Key results
- On Eyeful Tower, the method reaches 32.583 PSNR / 0.9328 SSIM / 0.1445 LPIPS, versus 31.854 PSNR for NeRF and 31.652 PSNR for Ref-NeRF.
- On the Shiny far-field dataset it reaches 26.564 PSNR / 0.7277 SSIM, versus 26.502 PSNR for Ref-NeRF.
- Ablation shows removing Gaussian optimization drops Eyeful Tower PSNR from 32.58 to 32.06, and the method is limited to blurry rather than perfect mirror reflections.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Gaussian directional encoding, Gaussian initialization, and joint Gaussian optimization | SpecNeRF · Eyeful Tower dataset | PSNR: 31.854 -> 32.583 | Helped |
| Use Gaussian directional encoding, Gaussian initialization, and joint Gaussian optimization | SpecNeRF · NISR + Inria dataset | PSNR: 30.748 -> 30.771 | Helped |
| Use Gaussian directional encoding, Gaussian initialization, and joint Gaussian optimization | SpecNeRF · Shiny dataset | PSNR: 26.502 -> 26.564 | Helped |
| Add Gaussian encoding initialization (ablation) | SpecNeRF · Eyeful Tower dataset | PSNR: 32.52 -> 32.58 | Helped |
| Jointly optimize Gaussians with NeRF (ablation) | SpecNeRF · Eyeful Tower dataset | PSNR: 32.06 -> 32.58 | Helped |
| Add monocular normal supervision (ablation) | SpecNeRF · Eyeful Tower dataset | PSNR: 32.31 -> 32.58 | Helped |
| Stop monocular normal supervision after 4K iterations (ablation) | SpecNeRF · Eyeful Tower dataset | PSNR: 32.46 -> 32.58 | Helped |
Surprising findings
- Without monocular normal supervision the Gaussian encoding can still 'cheat' plausible reflections even with erroneous normals.
Limitations
- Parameterizes specular color with only a few hundred Gaussians, so it cannot reproduce perfect mirror-like reflections and yields blurry reflections.
- Traversing all Gaussians per pixel is computationally costly, limiting scalability to more Gaussians.
- Monocular normals are error-prone, so supervision must be stopped early to avoid degrading reflections.
Experimental setup
Datasets: Eyeful Tower dataset NISR Inria Shiny dataset
Benchmarks: Eyeful Tower dataset NISR+Inria Shiny dataset
Metrics: PSNR SSIM LPIPS
Baselines: NeRF Ref-NeRF MS-NeRF NISR
Builds on: Ref-NeRF mip-NeRF 360 Nerfstudio nerfacto VR-NeRF
Closest papers by meaning
- Neural Directional Encoding for Efficient and Accurate View-Dependent Appearance Modeling (CVPR 2024 · similarity 0.92)
- GNeRP: Gaussian-guided Neural Reconstruction of Reflective Objects with Noisy Polarization Priors (ICLR 2024 · similarity 0.89)
- NeRFPrior: Learning Neural Radiance Field as a Prior for Indoor Scene Reconstruction (CVPR 2025 · similarity 0.87)
- MetaGS: A Meta-Learned Gaussian-Phong Model for Out-of-Distribution 3D Scene Relighting (NeurIPS 2025 · similarity 0.87)
- HyRF: Hybrid Radiance Fields for Memory-efficient and High-quality Novel View Synthesis (NeurIPS 2025 · similarity 0.87)
- A Pre-convolved Representation for Plug-and-Play Neural Illumination Fields (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.