Research map / 3D vision / Gaussian splatting
EnvGS: Modeling View-Dependent Appearance with Environment Gaussian
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes EnvGS, which models scene reflections with an explicit set of environment Gaussian primitives traced via a custom OptiX ray tracer over 2D Gaussians, blended with base Gaussians for real-time novel view synthesis of reflective scenes.
Problem
Real-time novel view synthesis of real-world scenes with complex high-frequency and near-field specular reflections, which SH-based 3DGS and distant-light environment-map methods struggle to model.
Approach
Replace environment maps with an explicit set of environment Gaussian primitives that are ray-traced along reflection directions using an OptiX-based differentiable 2DGS ray tracer, blended with base 2D Gaussians; joint optimization with monocular normal and perceptual losses.
Key results
- On Ref-Real, EnvGS reaches 24.617 PSNR / 0.241 LPIPS, the best among real-time methods (3DGS-DR 23.522, GaussianShader 22.875) and above NeRF-Casting's 24.670 PSNR-equivalent range in its class at ~26 FPS.
- On NeRF-Casting Shiny Scenes, EnvGS achieves 30.444 PSNR at 26.221 FPS, competitive with NeRF-Casting (31.023 PSNR) while being roughly 100x faster.
- On self-captured scenes, EnvGS improves PSNR on Audi to 28.629 versus 26.171 for 3DGS.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add environment Gaussian primitives as the reflection representation | EnvGS · Ref-Real | SSIM: 0.659 -> 0.671 | Helped |
| Add environment Gaussian primitives as the reflection representation | EnvGS · Audi | PSNR: 26.171 -> 28.629 | Helped |
| Add environment Gaussian primitives as the reflection representation | EnvGS · NeRF-Casting Scenes | PSNR: 31.740 -> 30.444 | Hurt |
| Add environment Gaussian primitives as the reflection representation | EnvGS · NeRF-Casting Scenes | LPIPS: 0.105 -> 0.148 | Hurt |
| Add joint optimization of base and environment Gaussians (ablation) | EnvGS · Ref-Real | PSNR: 24.034 -> 24.617 | Helped |
| Add monocular normal constraint (ablation) | EnvGS · Ref-Real | PSNR: 24.107 -> 24.617 | Helped |
| Use Gaussian environment representation instead of an environment map (ablation) | EnvGS · Ref-Real | PSNR: 24.145 -> 24.617 | Helped |
| Add color sabotage (ablation) | EnvGS · Ref-Real | PSNR: 24.268 -> 24.617 | Helped |
| Add normal propagation (ablation) | EnvGS · Ref-Real | PSNR: 24.192 -> 24.617 | Helped |
| Add perceptual loss (ablation) | EnvGS · Ref-Real | PSNR: 24.567 -> 24.617 | No clear effect |
Surprising findings
- Environment-map variant captures smooth distant reflections but fails on near-field and high-frequency details, so explicit Gaussian environment is preferred.
Limitations
- Cannot handle transparent and refractive materials since it only models reflection direction.
- Training time (2.5h) is longer than some real-time baselines such as 3DGS (0.6h).
Experimental setup
Datasets: Ref-Real NeRF-Casting Shiny Scenes Shiny Blender Mip-NeRF 360 self-captured real-world scenes
Benchmarks: Ref-Real NeRF-Casting Shiny Scenes Shiny Blender Mip-NeRF 360
Metrics: PSNR SSIM LPIPS FPS training time
Baselines: 3DGS 2DGS GaussianShader 3DGS-DR Ref-NeRF NeRF-Casting Zip-NeRF UniSDF ENVIDR NDE
Builds on: 2D Gaussian Splatting 3D Gaussian Splatting OptiX 3DGS-DR Ref-NeRF
Closest papers by meaning
- MaterialRefGS: Reflective Gaussian Splatting with Multi-view Consistent Material Inference (NeurIPS 2025 · similarity 0.89)
- Reflective Gaussian Splatting (ICLR 2025 · similarity 0.88)
- Decoupling Appearance Variations with 3D Consistent Features in Gaussian Splatting (AAAI 2025 · similarity 0.88)
- IBGS: Image-Based Gaussian Splatting (NeurIPS 2025 · similarity 0.87)
- FreeTimeGS: Free Gaussian Primitives at Anytime Anywhere for Dynamic Scene Reconstruction (CVPR 2025 · similarity 0.87)
- SVG-IR: Spatially-Varying Gaussian Splatting for Inverse Rendering (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.