Research map / 3D vision / Gaussian splatting
MS-GS: Multi-Appearance Sparse-View 3D Gaussian Splatting in the Wild
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MS-GS, a 3D Gaussian Splatting framework for sparse-view, multi-appearance in-the-wild scene reconstruction, combining semantic depth-aligned dense initialization with virtual-view geometry-guided pixel and feature supervision.
Problem
Novel view synthesis from few, multi-appearance in-the-wild photos is hard: sparse SfM initialization leads to oversmoothing and overfitting, and existing in-the-wild 3DGS/NeRF methods degrade badly with few views.
Approach
Densify the SfM initialization by aligning monocular depth to SfM points within SAM-prompted local semantic masks, then regularize training with virtual-view 3D warping supervision at pixel and VGG feature levels, using per-image appearance embeddings and per-Gaussian feature embeddings.
Key results
- On the sparse unbounded drone dataset, MS-GS reaches 19.87 PSNR and 0.580 SSIM versus the best prior method GS-W at 17.33 PSNR and 0.491 SSIM.
- On sparse Mip-NeRF 360 with GT poses, MS-GS reports 22.39 PSNR and 0.702 SSIM, above the best baseline FSGS at 21.67 PSNR.
- MS-GS trains in 0.29 GPU hours at 373 FPS on the drone dataset, versus 1.01 GPU hours for Wild-GS.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MS-GS instead of SPARS3R | Sparse-view 3DGS reconstruction · Sparse Mip-NeRF 360 | PSNR: 21.48 -> 22.39 | Helped |
| Use MS-GS instead of GS-W | Sparse-view multi-appearance reconstruction · Sparse Unbounded Drone | PSNR: 17.33 -> 19.87 | Helped |
| Use MS-GS instead of NeRF-W | Sparse-view multi-appearance reconstruction · Sparse Phototourism | PSNR: 17.93 -> 18.99 | Helped |
| Add semantic dense initialization to the sparse SfM baseline (ablation) | 3DGS with per-image and per-Gaussian appearance embeddings · Sparse Unbounded Drone | PSNR: 18.49 -> 19.29 | Helped |
| Add pixel warping loss to dense initialization (ablation) | MS-GS · Sparse Unbounded Drone | PSNR: 19.29 -> 19.63 | Helped |
| Add geometry-guided feature loss to dense initialization (ablation) | MS-GS · Sparse Unbounded Drone | PSNR: 19.29 -> 19.65 | Helped |
| Add both pixel warping and feature losses to dense initialization (ablation) | MS-GS · Sparse Unbounded Drone | PSNR: 19.29 -> 19.87 | Helped |
| Replace sparse SfM initialization with image-level depth alignment (ablation) | 3DGS initialization · Sparse Mip-NeRF 360 | PSNR: 20.83 -> 21.06 | Helped |
| Use semantic depth alignment instead of image-level alignment (ablation) | 3DGS initialization · Sparse Mip-NeRF 360 | PSNR: 21.06 -> 21.96 | Helped |
| Replace semantic depth alignment with DUSt3R initialization (ablation) | 3DGS initialization · Sparse Mip-NeRF 360 | PSNR: 21.96 -> 19.89 | Hurt |
| Initialize per-image appearance embeddings near zero instead of with a normal distribution in [0,1] (ablation) | MS-GS appearance embeddings · Sparse Unbounded Drone | PSNR: 18.77 -> 19.29 | Helped |
| Add dense initialization to GS-W (ablation) | GS-W · Sparse Unbounded Drone | PSNR: 17.33 -> 18.23 | Helped |
Surprising findings
- Initializing near zero for per-image appearance embeddings outperformed the common normal-distribution initialization.
- Applying the dense semantic initialization to existing in-the-wild methods (GS-W, WildGaussians) improved them substantially but they remained far below MS-GS.
Limitations
- Not designed for transient objects, which remain under-constrained under sparse views.
- Struggles with non-Lambertian surfaces; specular highlights are smoothed out.
- Regions without dense initialization can still produce artifacts.
Experimental setup
Datasets: Sparse Mip-NeRF 360 Sparse Phototourism (Brandenburg Gate, Sacre Coeur, Trevi Fountain) Sparse Unbounded Drone Dataset
Benchmarks: Sparse Mip-NeRF 360 Sparse Phototourism Sparse Unbounded Drone Dataset
Metrics: PSNR SSIM LPIPS DreamSim
Baselines: DRGS DNGS SparseGS FSGS SPARS3R NeRF-W Ha-NeRF CR-NeRF GS-W Wild-GS WildGaussians DUSt3R
Builds on: 3D Gaussian Splatting Splatfacto NeRF-W FSGS SPIn-NeRF-style point-prompted segmentation
Closest papers by meaning
- Wild-GS: Real-Time Novel View Synthesis from Unconstrained Photo Collections (NeurIPS 2024 · similarity 0.92)
- Generative Sparse-View Gaussian Splatting (CVPR 2025 · similarity 0.92)
- Quantifying and Alleviating Co-Adaptation in Sparse-View 3D Gaussian Splatting (NeurIPS 2025 · similarity 0.91)
- FewViewGS: Gaussian Splatting with Few View Matching and Multi-stage Training (NeurIPS 2024 · similarity 0.91)
- GS-IR: 3D Gaussian Splatting for Inverse Rendering (CVPR 2024 · similarity 0.91)
- Horizon-GS: Unified 3D Gaussian Splatting for Large-Scale Aerial-to-Ground Scenes (CVPR 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.