Research map / 3D vision / Gaussian splatting
3D Gaussian Inpainting with Depth-Guided Cross-View Consistency
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes 3DGIC, a 3D Gaussian Splatting inpainting framework that uses rendered depth to refine inpainting masks by back-projecting background visible from other views, then optimizes new Gaussians with cross-view LPIPS consistency from a reference-view 2D inpainting.
Problem
Object removal and inpainting in 3D scenes represented by NeRF or 3DGS suffers from cross-view texture and geometry inconsistency, partly because per-view 2D inpainting masks cover regions visible from other views.
Approach
Use rendered depth maps from all training views to back-project visible background pixels into the reference view, shrinking the inpainting mask to only regions unseen in any view; then supervise new 3DGS Gaussians with a reference-view 2D inpainting plus a cross-view LPIPS loss.
Key results
- 3DGIC with LDM as 2D inpainter achieves FID 36.4 and m-FID 96.3 on SPIn-NeRF, the best on all four metrics versus GScream (38.6, 101.6).
- 3DGIC with LaMa inpainter achieves FID 41.7 and m-FID 102.4, still outperforming MVIP-NeRF and MALD-NeRF.
- Ablation on the Bear scene shows that using original SAM masks changes visible background, and that cross-view consistency loss is needed for consistent inpainting.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use depth-guided masks and cross-view refinement instead of GScream | 3DGIC (LDM) · SPIn-NeRF | FID: 38.6 -> 36.4 | Helped |
| Use depth-guided masks and cross-view refinement instead of GScream | 3DGIC (LDM) · SPIn-NeRF | m-FID: 101.6 -> 96.3 | Helped |
| Use depth-guided masks and cross-view refinement instead of GScream | 3DGIC (LDM) · SPIn-NeRF | LPIPS: 0.28 -> 0.26 | Helped |
| Use depth-guided masks and cross-view refinement instead of GScream | 3DGIC (LDM) · SPIn-NeRF | m-LPIPS: 0.033 -> 0.028 | Helped |
| Add depth-guided masks to replace the original object masks as inpainting masks (ablation) | 3DGIC · Bear scene (InNeRF360) | visible-background preservation | Helped |
| Add inpainting-guided 3DGS refinement with cross-view consistency instead of fine-tuning on all 2D inpainting results (ablation) | 3DGIC · Bear scene (InNeRF360) | cross-view consistency | Helped |
| Add depth-guided masks and inpainting-guided 3DGS refinement (ablation) | 3DGIC · Bear scene (InNeRF360) | rendering fidelity | Helped |
Limitations
- Evaluated mainly on the SPIn-NeRF dataset, whose camera poses cover only a small range near the front view.
- Depends on the quality of the input object masks (e.g., from SAM) and on the 2D inpainter used for reference-view supervision.
- Ablation and 360-degree scenes are only evaluated qualitatively.
Experimental setup
Datasets: SPIn-NeRF dataset InNeRF360 Mip-NeRF360 LeRF
Benchmarks: SPIn-NeRF dataset
Metrics: FID m-FID LPIPS m-LPIPS
Baselines: SPIn-NeRF MVIP-NeRF Gaussian Grouping MALD-NeRF GScream
Builds on: 3D Gaussian Splatting Gaussian Grouping SPIn-NeRF LaMa latent diffusion inpainting
Closest papers by meaning
- RNG: Relightable Neural Gaussians (CVPR 2025 · similarity 0.87)
- GaussianSpa: An "Optimizing-Sparsifying" Simplification Framework for Compact and High-Quality 3D Gaussian Splatting (CVPR 2025 · similarity 0.87)
- Gaussian Herding across Pens: An Optimal Transport Perspective on Global Gaussian Reduction for 3DGS (NeurIPS 2025 · similarity 0.87)
- GaussianPro: 3D Gaussian Splatting with Progressive Propagation (ICML 2024 · similarity 0.87)
- 3D-HGS: 3D Half-Gaussian Splatting (CVPR 2025 · similarity 0.87)
- IMFine: 3D Inpainting via Geometry-guided Multi-view Refinement (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.