Research map / 3D vision / Gaussian splatting
Robust Neural Rendering in the Wild with Asymmetric Dual 3D Gaussian Splatting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Trains two 3DGS models with a mutual consistency loss and divergent masks (multi-cue adaptive hard mask and self-supervised soft mask), plus a lightweight Dynamic EMA proxy variant, for robust in-the-wild scene reconstruction.
Problem
Reconstructing 3D scenes from in-the-wild photos is corrupted by transient distractors and varying illumination, causing stochastic artifacts and unstable reconstructions when heuristics are used.
Approach
Exploit the run-to-run randomness of artifacts: train two 3DGS models with an L1 mutual consistency loss on renderings, use asymmetric (complementary) masks to reduce confirmation bias, and replace one model with a dynamic EMA proxy (EMA accounting for densification/pruning) with alternating masks to save compute.
Key results
- On NeRF On-the-go, GS-GS reaches 24.34 PSNR in high occlusion vs 23.05 for HybridGS (best prior).
- On RobustNeRF, GS-GS achieves 36.18 PSNR on Crab and 37.18 on Yoda, exceeding the best prior by about 1 dB in complex scenes.
- EMA-GS cuts PhotoTourism training from 7.2 hours (WildGaussian) to 2.9 hours while improving PSNR on the Brandenburg Gate and Sacre Coeur scenes.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use dual 3DGS models with mutual consistency instead of a single model (ablation) | AsymGS · NeRF On-the-go | PSNR | Helped |
| Add mutual consistency loss to EMA-GS (ablation) | EMA-GS · NeRF On-the-go | PSNR: 23.10 -> 23.40 | Helped |
| Add mutual consistency loss to GS-GS (ablation) | GS-GS · NeRF On-the-go | PSNR: 23.13 -> 23.61 | Helped |
| Add masking instead of training without masks in EMA-GS (ablation) | EMA-GS · NeRF On-the-go | PSNR: 21.68 -> 23.40 | Helped |
| Add masking instead of training without masks in GS-GS (ablation) | GS-GS · NeRF On-the-go | PSNR: 22.39 -> 23.61 | Helped |
| Add the self-supervised soft mask (ablation) | GS-GS · NeRF On-the-go | PSNR: 23.32 -> 23.61 | Helped |
| Add the multi-cue adaptive mask (ablation) | GS-GS · NeRF On-the-go | PSNR: 22.76 -> 23.61 | Helped |
| Add the self-supervised soft mask (ablation) | EMA-GS · NeRF On-the-go | PSNR: 22.94 -> 23.40 | Helped |
| Add the multi-cue adaptive mask (ablation) | EMA-GS · NeRF On-the-go | PSNR: 22.15 -> 23.40 | Helped |
| Replace alternating masks with Mixup regularization (ablation) | EMA-GS · NeRF On-the-go | PSNR: 23.40 -> 23.39 | No clear effect |
| Replace alternating masks with Gaussian dropout (ablation) | EMA-GS · NeRF On-the-go | PSNR: 23.40 -> 23.39 | No clear effect |
| Use GS-GS instead of the EMA proxy variant | AsymGS · NeRF On-the-go | PSNR: 23.40 -> 23.61 | Helped |
Surprising findings
- Using the same mask type for both models (or no masks) degrades the EMA and dual models, showing masks are necessary for diversity.
- Dual-model ensemble without the mutual consistency loss drops about 0.5 dB PSNR, so the consistency term rather than ensembling drives gains.
Limitations
- Appearance modeling struggles to capture fine-grained effects such as object highlights, likely due to limited training data diversity.
- Dual-model training doubles computation relative to single 3DGS, motivating the EMA variant.
- Error bars are not reported in the main paper; five-run statistics are only in the appendix.
Experimental setup
Datasets: NeRF On-the-go RobustNeRF PhotoTourism
Benchmarks: NeRF On-the-go RobustNeRF PhotoTourism
Metrics: PSNR SSIM LPIPS training hours
Baselines: 3DGS Mip-Splatting GS-W WildGaussian HybridGS SLS-mlp NeRF On-the-go RobustNeRF SWAG NeRF-W
Builds on: 3D Gaussian Splatting Mip-Splatting WildGaussian SAM DINOv2
Closest papers by meaning
- WildGaussians: 3D Gaussian Splatting In the Wild (NeurIPS 2024 · similarity 0.91)
- MaskGaussian: Adaptive 3D Gaussian Representation from Probabilistic Masks (CVPR 2025 · similarity 0.91)
- DOGS: Distributed-Oriented Gaussian Splatting for Large-Scale 3D Reconstruction Via Gaussian Consensus (NeurIPS 2024 · similarity 0.90)
- LP-3DGS: Learning to Prune 3D Gaussian Splatting (NeurIPS 2024 · similarity 0.90)
- GaussianPro: 3D Gaussian Splatting with Progressive Propagation (ICML 2024 · similarity 0.90)
- Learning Efficient Fuse-and-Refine for Feed-Forward 3D Gaussian Splatting (NeurIPS 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.