Research map / 3D vision / Gaussian splatting
Normal-GS: 3D Gaussian Splatting with Normal-Involved Rendering
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Normal-GS re-parameterizes 3DGS colors as the dot product of surface normals and an anchor-based, locally shared Integrated Directional Illumination Vector (IDIV), plus IDE specular terms, to improve both rendering quality and normal accuracy.
Problem
3D Gaussian Splatting rendering is disconnected from surface normals, so attempts to regularize normals degrade rendering quality; the appearance-geometry trade-off ("seesaw") limits accurate surface estimation in 3DGS.
Approach
Model diffuse color as albedo times dot product of the surface normal with a learned Integrated Directional Illumination Vector derived from the rendering equation, with IDIVs encoded implicitly via Scaffold-GS anchor features decoded by MLPs, and specular color via Integrated Directional Encoding on the reflection direction.
Key results
- On Mip-NeRF 360, Normal-GS reports PSNR 29.341, SSIM 0.869, LPIPS 0.194, comparable to SpecGaussian (29.287 PSNR) and ScaffoldGS.
- On Tanks & Temples, Normal-GS achieves the best LPIPS (0.174) and PSNR 24.219 vs SpecGaussian 24.502.
- On NeRF Synthetic normal estimation, Normal-GS achieves the lowest normal MAE of 20.71 vs GShader 23.56, ScaffoldGS 25.56, SpecGaussian 45.98.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use explicit normal-involved shading with IDIVs instead of Scaffold-GS color prediction | Normal-GS / Scaffold-GS · Mip-NeRF 360 | PSNR: 29.267 -> 29.341 | Helped |
| Use explicit normal-involved shading with IDIVs instead of Scaffold-GS color prediction | Normal-GS / Scaffold-GS · Tanks & Temples | PSNR: 24.088 -> 24.219 | Mixed |
| Use explicit normal-involved shading with IDIVs instead of Scaffold-GS color prediction | Normal-GS / Scaffold-GS · Deep Blending | PSNR: 30.140 -> 30.187 | Helped |
| Add normal inputs to the Scaffold-GS color MLP | Scaffold-GS · Mip-NeRF 360 | PSNR: 29.267 -> 29.177 | Hurt |
| Add normal inputs to the Scaffold-GS color MLP | Scaffold-GS · Tanks & Temples | PSNR: 24.088 -> 23.976 | Hurt |
| Add normal inputs to the Scaffold-GS color MLP | Scaffold-GS · Deep Blending | PSNR: 30.140 -> 30.163 | Helped |
| Add IDIV-based normal involvement to color modeling (ablation) | 3DGS · Rendering quality and normal estimation (ablation; benchmark not specified) | Rendering quality and normal smoothness | Helped |
| Replace loss-weight-sensitive IDIV regularization with anchor-based local IDIV prediction (ablation) | Normal-GS · Rendering quality and normal estimation (ablation; benchmark not specified) | Solution-space regularization | Helped |
| Add the specular component to the IDIV-based model (ablation) | Normal-GS · Rendering quality (ablation; benchmark not specified) | Rendering quality | Helped |
| Use the self-regularized normal loss on distant outdoor scenes | Normal-GS · Distant outdoor scenes | Rendering quality | Hurt |
Surprising findings
- Simply feeding normals into the Scaffold-GS color MLP (ScaffoldGS w/ N) does not improve rendering and can slightly degrade it, indicating implicit normal use is insufficient.
Limitations
- Self-regularized depth-normal loss is suboptimal for distant outdoor scenes, degrading rendering quality there.
- Model performance is sensitive to loss weights when IDIVs are stored per Gaussian without anchor-based regularization.
- Per-scene optimization is required, with computational cost concerns.
Experimental setup
Datasets: NeRF Synthetic Mip-NeRF 360 Tanks and Temples Deep Blending
Benchmarks: Mip-NeRF 360 Tanks and Temples Deep Blending NeRF Synthetic
Metrics: PSNR SSIM LPIPS Mean Angular Error (normal)
Baselines: 3DGS Scaffold-GS ScaffoldGS w/ N GaussianShader SpecGaussian
Builds on: 3D Gaussian Splatting Scaffold-GS Ref-NeRF 2D Gaussian Splatting GaussianShader
Closest papers by meaning
- PUP 3D-GS: Principled Uncertainty Pruning for 3D Gaussian Splatting (CVPR 2025 · similarity 0.91)
- Spec-Gaussian: Anisotropic View-Dependent Appearance for 3D Gaussian Splatting (NeurIPS 2024 · similarity 0.91)
- IBGS: Image-Based Gaussian Splatting (NeurIPS 2025 · similarity 0.91)
- Speedy-Splat: Fast 3D Gaussian Splatting with Sparse Pixels and Sparse Primitives (CVPR 2025 · similarity 0.91)
- VCR-GauS: View Consistent Depth-Normal Regularizer for Gaussian Surface Reconstruction (NeurIPS 2024 · similarity 0.91)
- GS-IR: 3D Gaussian Splatting for Inverse Rendering (CVPR 2024 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.