Research map / 3D vision / Gaussian splatting
3DGS-Enhancer: Enhancing Unbounded 3D Gaussian Splatting with View-consistent 2D Diffusion Priors
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Enhances sparse-view or low-quality unbounded 3DGS renderings by restoring novel views with a fine-tuned video diffusion model (SVD-based) conditioned on reference views, then fine-tunes 3DGS on the enhanced views with confidence weighting.
Problem
Novel views rendered from 3D Gaussian splatting reconstructions of unbounded scenes with sparse input views contain artifacts (ellipsoid-like, hollow regions) due to under-sampled areas, and generative priors often lack 3D view consistency.
Approach
Reformulate 3D-consistent image restoration as temporally consistent video interpolation: a video LDM (SVD) takes rendered frames bracketed by two reference views, a spatial-temporal decoder with controllable feature warping and color correction decodes the latents, and the enhanced views are used to fine-tune 3DGS with image-level and pixel-level (scale-based) confidence weighting.
Key results
- On DL3DV with 3 views, 3DGS-Enhancer reaches 14.33 PSNR vs 10.97 for 3DGS and 11.10 for DNGaussian.
- On DL3DV with 9 views, PSNR 18.50 / SSIM 0.630 / LPIPS 0.305, outperforming 3DGS (14.99 PSNR).
- Cross-dataset on Mip-NeRF 360 with 9 views: 16.22 PSNR vs 12.65 for 3DGS when trained on DL3DV.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use 3DGS-Enhancer instead of the strongest listed baseline | Few-shot NVS · DL3DV, 3 views | PSNR: 11.46 -> 14.33 | Helped |
| Use 3DGS-Enhancer instead of the strongest listed baseline | Few-shot NVS · DL3DV, 6 views | PSNR: 13.34 -> 16.94 | Helped |
| Use 3DGS-Enhancer instead of the strongest listed baseline | Few-shot NVS · DL3DV, 9 views | PSNR: 14.99 -> 18.50 | Helped |
| Use 3DGS-Enhancer instead of the strongest listed baseline | Few-shot NVS · Mip-NeRF360, 6 views | PSNR: 13.08 -> 13.96 | Helped |
| Use 3DGS-Enhancer instead of the strongest listed baseline | Few-shot NVS · Mip-NeRF360, 9 views | PSNR: 13.73 -> 16.22 | Helped |
| Add real images as reference views during 3DGS fine-tuning (ablation) | 3DGS-Enhancer · DL3DV, 3/6/9/12 views | PSNR: 14.33 -> 17.01 | Helped |
| Add image-level confidence weighting (ablation) | 3DGS-Enhancer with real-image references · DL3DV, 3/6/9/12 views | PSNR: 17.01 -> 17.29 | Helped |
| Add pixel-level confidence weighting (ablation) | 3DGS-Enhancer with real-image references · DL3DV, 3/6/9/12 views | PSNR: 17.01 -> 17.16 | Helped |
| Use image-level and pixel-level confidence weighting together (ablation) | 3DGS-Enhancer with real-image references · DL3DV, 3/6/9/12 views | PSNR: 17.29 -> 17.34 | Helped |
| Add STD temporal layers (ablation) | 3DGS-Enhancer · DL3DV, 9 views | PSNR: 18.11 -> 18.44 | Helped |
| Add color correction (ablation) | 3DGS-Enhancer with STD temporal layers · DL3DV, 9 views | PSNR: 18.44 -> 18.50 | Helped |
Surprising findings
- Feeding the original real input views (rather than only diffusion outputs) to 3DGS fine-tuning gives the largest ablation gain (14.33 to 17.01 PSNR).
Limitations
- Relies on adjacent views for interpolation, so it cannot be directly applied to single-view generation.
- Confidence-aware fine-tuning strategies are relatively simple.
- Per-scene pipeline is slower than vanilla 3DGS (24.5 min vs 10.5 min).
Experimental setup
Datasets: DL3DV-10K Mip-NeRF 360
Benchmarks: DL3DV Mip-NeRF 360 LLFF
Metrics: PSNR SSIM LPIPS
Baselines: Mip-NeRF RegNeRF FreeNeRF 3DGS DNGaussian Zip-NeRF ZeroNVS ReconFusion
Builds on: 3D Gaussian splatting Stable Video Diffusion ActiveNeRF StableSR Controllable Feature Warping
Closest papers by meaning
- Generative Sparse-View Gaussian Splatting (CVPR 2025 · similarity 0.92)
- Quantifying and Alleviating Co-Adaptation in Sparse-View 3D Gaussian Splatting (NeurIPS 2025 · similarity 0.92)
- Taming Video Diffusion Prior with Scene-Grounding Guidance for 3D Gaussian Splatting from Sparse Inputs (CVPR 2025 · similarity 0.92)
- LP-3DGS: Learning to Prune 3D Gaussian Splatting (NeurIPS 2024 · similarity 0.91)
- FlexGS: Train Once, Deploy Everywhere with Many-in-One Flexible 3D Gaussian Splatting (CVPR 2025 · similarity 0.91)
- DropGaussian: Structural Regularization for Sparse-view Gaussian Splatting (CVPR 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.