Research map / 3D vision / Gaussian splatting
WildGaussians: 3D Gaussian Splatting In the Wild
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Extends 3D Gaussian Splatting to in-the-wild photo collections by embedding trainable per-image and per-Gaussian appearance vectors in a small affine color-toning MLP and adding a DINOv2-cosine-based uncertainty predictor that masks occluders.
Problem
3DGS struggles with in-the-wild captures that have appearance/illumination changes and transient occluders, since its explicit per-Gaussian colors cannot be conditioned on image appearance cheaply and its densification is sensitive to incorrect training signals.
Approach
Per-Gaussian and per-image embeddings feed a small MLP that predicts an affine color transform applied to the SH base color (baked back into 3DGS after training for fast rendering), plus an uncertainty predictor trained with a patch-wise DINOv2 cosine-similarity loss and converted into a binary mask so gradient statistics for densification are preserved.
Key results
- On Photo Tourism, WildGaussians reaches 27.77 PSNR on Brandenburg Gate and 22.56 on Sacre Coeur, outperforming SWAG and GS-W while rendering at 117 FPS.
- On NeRF On-the-go, average PSNR of 22.15 vs 21.71 for NeRF On-the-go, with rendering at 108 FPS versus under 1 FPS for NeRF On-the-go.
- Ablation on Photo Tourism shows removing appearance modeling drops average PSNR substantially (e.g., Brandenburg Gate 27.77 to 20.81).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add appearance modeling with per-image and per-Gaussian embeddings (ablation) | 3DGS · Photo Tourism | PSNR: 20.81 -> 27.77; 17.19 -> 22.56; 17.42 -> 23.63 | Helped |
| Add DINO-based uncertainty modeling (ablation) | WildGaussians · NeRF On-the-go high occlusion | PSNR: 20.27 -> 23.03 | Helped |
| Add uncertainty modeling (ablation) | WildGaussians · Photo Tourism — Brandenburg Gate | PSNR: 27.16 -> 27.77 | Helped |
| Add uncertainty modeling (ablation) | WildGaussians · Photo Tourism — Sacre Coeur | PSNR: 21.97 -> 22.56 | Helped |
| Add uncertainty modeling (ablation) | WildGaussians · Photo Tourism — Trevi Fountain | PSNR: 23.82 -> 23.63 | Hurt |
| Add appearance modeling (ablation) | WildGaussians · NeRF On-the-go low occlusion | PSNR: 20.80 -> 20.62 | Hurt |
| Add appearance modeling (ablation) | WildGaussians · NeRF On-the-go high occlusion | PSNR: 22.80 -> 23.03 | Helped |
| Initialize Gaussian embeddings with Fourier features (ablation) | WildGaussians · Photo Tourism — Brandenburg Gate | PSNR: 27.33 -> 27.77 | Helped |
| Use per-Gaussian embeddings rather than only image-level appearance conditioning (ablation) | WildGaussians · Photo Tourism — Sacre Coeur | PSNR: 18.68 -> 22.56 | Helped |
| Replace DINO uncertainty with MSSIM uncertainty (ablation) | WildGaussians · Photo Tourism — Sacre Coeur | PSNR: 21.19 -> 22.56 | Hurt |
| Use explicit segmentation masks instead of learned uncertainty (ablation) | WildGaussians · Photo Tourism — Brandenburg Gate | PSNR: 26.87 -> 27.77 | Hurt |
| Use WildGaussians instead of NeRF On-the-go | NeRF On-the-go benchmark · NeRF On-the-go average | PSNR: 21.71 -> 22.15 | Helped |
Surprising findings
- Disabling uncertainty modeling slightly hurts or does not help on the low-occlusion On-the-go dataset, and on Trevi Fountain uncertainty does not improve performance.
Limitations
- Cannot capture highlights on objects (specular effects).
- Regions occluded in nearly all training images are hard to reconstruct.
- Uncertainty modeling still struggles in some challenging scenarios.
Experimental setup
Datasets: NeRF On-the-go Photo Tourism MipNeRF360
Benchmarks: NeRF On-the-go Photo Tourism MipNeRF360
Metrics: PSNR SSIM LPIPS GPU hours FPS
Baselines: NeRF On-the-go 3DGS Mip-Splatting Gaussian Opacity Fields GS-W SWAG NeRF-W K-Planes
Builds on: 3D Gaussian Splatting NeRF-W NeRF On-the-go Mip-Splatting DINOv2
Closest papers by meaning
- Robust Neural Rendering in the Wild with Asymmetric Dual 3D Gaussian Splatting (NeurIPS 2025 · similarity 0.91)
- MaskGaussian: Adaptive 3D Gaussian Representation from Probabilistic Masks (CVPR 2025 · similarity 0.91)
- VastGaussian: Vast 3D Gaussians for Large Scene Reconstruction (CVPR 2024 · similarity 0.90)
- FlashGS: Efficient 3D Gaussian Splatting for Large-scale and High-resolution Rendering (CVPR 2025 · similarity 0.90)
- HybridGS: Decoupling Transients and Statics with 2D and 3D Gaussian Splatting (CVPR 2025 · similarity 0.90)
- GaussianPro: 3D Gaussian Splatting with Progressive Propagation (ICML 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.