Research map / 3D vision / Gaussian splatting
Spec-Gaussian: Anisotropic View-Dependent Appearance for 3D Gaussian Splatting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Spec-Gaussian replaces spherical harmonics in 3D Gaussian splatting with an anisotropic spherical Gaussian appearance field, plus a coarse-to-fine training scheme and anchor-based acceleration, to model specular and anisotropic appearance and reduce floaters.
Problem
3D Gaussian splatting struggles to model specular highlights and anisotropic view-dependent appearance because low-order spherical harmonics cannot represent high-frequency signals, and it overfits with floaters in real-world scenes.
Approach
Model each 3D Gaussian's view-dependent appearance with N=32 anisotropic spherical Gaussians whose outputs are latent features decoded by a small MLP into specular color (with diffuse color from SH), query them with the reflection direction using an unsupervised shortest-axis normal, and train coarse-to-fine with resolution ramp-up and L1 gradient accumulation for densification.
Key results
- On Mip-NeRF 360, the full Spec-Gaussian reaches PSNR 28.18, SSIM 0.835, LPIPS 0.176 overall, versus 27.79 PSNR for 3D-GS, at 33 FPS and 847MB.
- On NSVF synthetic, Ours averages 38.40 PSNR versus 37.07 for 3D-GS and 37.80 for NeuRBF.
- On NeRF synthetic, Ours averages 34.19 PSNR and 0.971 SSIM, compared with 33.32 PSNR for 3D-GS.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace SH appearance with an ASG appearance field | 3D-GS · Anisotropic Synthetic | PSNR: 33.81 -> 37.70 | Helped |
| Add coarse-to-fine training (ablation) | 3D-GS on Mip-NeRF 360 · Mip-NeRF 360 | rendering quality | Helped |
| Add an L1 constraint to gradients used for densification (ablation) | 3D-GS coarse-to-fine training · Real-world scenes | number of floaters | Helped |
| Add normals to the ASG appearance field (ablation) | Spec-Gaussian · NeRF synthetic | rendering metrics | Helped |
| Use ASG directly to output color instead of decoding ASG features with an MLP (ablation) | ASG appearance field · Specular and anisotropic scenes | rendering quality | Hurt |
| Replace ASG with higher-order SH (6-order) (ablation) | 3D-GS · Specular and anisotropic scenes | rendering quality | No clear effect |
| Increase the number of MLP layers to 4 (ablation) | 3D-GS and Scaffold-GS · Specular and anisotropic scenes | rendering quality | No clear effect |
| Separate diffuse and specular color modeling (ablation) | ASG appearance field · High-frequency appearance | fitting ability | Helped |
Surprising findings
- Directly using ASG to output color fails to model specular and anisotropic components; a feature-decoupling MLP on ASG latent features is needed.
- Higher-order SH (6-order) and deeper MLPs (4 layers) do not help 3D-GS and Scaffold-GS, per the authors.
Limitations
- Cannot differentiate reflections from material properties because there is no explicit geometry; reflections remain unmodeled.
- Normals from the shortest Gaussian axis are unsupervised and not physically accurate.
- The full model is slower (33 FPS) and larger (847MB) than vanilla 3D-GS on Mip-NeRF 360.
Experimental setup
Datasets: NeRF synthetic NSVF synthetic Mip-NeRF 360 Anisotropic Synthetic (self-rendered)
Benchmarks: NeRF synthetic NSVF synthetic Mip-NeRF 360 Anisotropic Synthetic
Metrics: PSNR SSIM LPIPS (VGG) FPS memory
Baselines: 3D-GS Scaffold-GS GaussianShader NSVF TensoRF NeuRBF Tri-MipRF iNGP
Builds on: 3D Gaussian splatting Scaffold-GS anisotropic spherical gaussian Ref-NeRF
Closest papers by meaning
- IBGS: Image-Based Gaussian Splatting (NeurIPS 2025 · similarity 0.93)
- GaussianPro: 3D Gaussian Splatting with Progressive Propagation (ICML 2024 · similarity 0.92)
- 3D-HGS: 3D Half-Gaussian Splatting (CVPR 2025 · similarity 0.91)
- Textured Gaussians for Enhanced 3D Scene Appearance Modeling (CVPR 2025 · similarity 0.91)
- Normal-GS: 3D Gaussian Splatting with Normal-Involved Rendering (NeurIPS 2024 · similarity 0.91)
- SpecTRe-GS: Modeling Highly Specular Surfaces with Reflected Nearby Objects by Tracing Rays in 3D Gaussian Splatting (CVPR 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.