Research map / 3D vision / Gaussian splatting
RNG: Relightable Neural Gaussians
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Relightable Neural Gaussians, a 3DGS-based framework that models view- and light-dependent reflectance with latent features decoded by an MLP, adds a shadow cue with depth refinement, and uses a two-stage forward-deferred training to relight objects with hard surfaces or soft boundaries.
Problem
Relighting 3D assets with ill-defined shapes (fur, fabric) and hard surfaces without relying on surface constraints or analytic shading models, while keeping training and rendering fast and shadows sharp under point lights.
Approach
Each Gaussian carries a latent feature vector decoded by an MLP conditioned on view direction, light direction, and a shadow cue obtained by a 3DGS-based shadow mapping pass with a learned depth refinement; training switches from forward shading to deferred shading to balance geometry and shadow quality.
Key results
- Average PSNR 31.84 / SSIM 0.9475 / LPIPS 0.0392 across 14 point-light scenes, versus NRHints 31.43 and GS3 30.54 (Table 1).
- Under novel environment lighting, average PSNR 30.70 versus 25.27 for GS-IR and 24.04 for Relightable 3D Gaussian (Table 2).
- Training takes about 1.3 hours and renders at 60 fps on an RTX 4090, more than 20x faster to train than NRHints.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use RNG instead of NRHints | Point-light relighting · Point-light scenes (average) | PSNR: 31.4266 -> 31.8352 | Helped |
| Use RNG instead of GS3 | Point-light relighting · Point-light scenes (average) | LPIPS: 0.0478 -> 0.0392 | Helped |
| Use RNG instead of GS-IR | Environment-light relighting · Environment-light scenes (average) | PSNR: 25.2726 -> 30.6981 | Helped |
| Use RNG instead of RelightableGS | Environment-light relighting · Environment-light scenes (average) | LPIPS: 0.0623 -> 0.0557 | Helped |
| Add depth refinement to the shadow-cue pipeline (ablation) | RNG · Shadow mapping | shadow-cue accuracy | Helped |
| Add the shadow cue to the decoder (ablation) | RNG · Relighting | shadow quality | Helped |
| Use deferred shading in the second training stage (ablation) | RNG · Relighting | shadow quality | Helped |
Surprising findings
- NRHints achieves higher PSNR than RNG on several scenes, attributed to its SDF prior and larger network.
- Forward shading yields better geometry but worse shadows, while deferred shading improves shadows but causes floaters.
Limitations
- Shadow quality is bounded by geometry reconstruction accuracy for soft-boundary objects.
- Rasterization without ray marching makes highly reflective appearances blurry.
- Less PSNR accuracy than NRHints on some surface-like scenes.
Experimental setup
Datasets: NRHints dataset RNA dataset
Benchmarks: NRHints dataset RNA dataset
Metrics: PSNR SSIM LPIPS
Baselines: NRHints GS3 GS-IR Relightable 3D Gaussian
Builds on: 3D Gaussian Splatting NRHints Deferred Neural Relighting
Closest papers by meaning
- MetaGS: A Meta-Learned Gaussian-Phong Model for Out-of-Distribution 3D Scene Relighting (NeurIPS 2025 · similarity 0.91)
- Compact 3D Gaussian Representation for Radiance Field (CVPR 2024 · similarity 0.90)
- Gaussian Herding across Pens: An Optimal Transport Perspective on Global Gaussian Reduction for 3DGS (NeurIPS 2025 · similarity 0.90)
- GaussianPro: 3D Gaussian Splatting with Progressive Propagation (ICML 2024 · similarity 0.90)
- LP-3DGS: Learning to Prune 3D Gaussian Splatting (NeurIPS 2024 · similarity 0.90)
- Gaussian Shadow Casting for Neural Characters (CVPR 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.