Research map / 3D vision / 3D generation
AuraFusion360: Augmented Unseen Region Alignment for Reference-based 360deg Unbounded Scene Inpainting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AuraFusion360, a reference-based 360° unbounded scene inpainting method for Gaussian Splatting using depth-warping unseen masks, zero-shot Adaptive Guided Depth Diffusion, and SDEdit with DDIM inversion, plus a new 360-USID dataset.
Problem
Object removal and hole filling in 360° unbounded 3D scenes suffer from multi-view inconsistency and geometric inaccuracy, especially when unseen regions must be extrapolated under large viewpoint changes.
Approach
Uses depth warping across views to produce bounding-box prompts for SAM2 that identify truly unseen regions, aligns monocular depth to the incomplete reference-view depth via an adaptive loss inside Marigold's denoising loop (AGDD), and refines rendered Gaussians with SDEdit using DDIM-inverted noise conditioned on the reference view.
Key results
- On 360-USID, AuraFusion360 achieves average PSNR 17.661 and LPIPS 0.388, the best of all compared methods (next best 2DGS+LeftRefill at 16.282 PSNR / 0.518 LPIPS).
- Ablation: SDEdit strength 0.85 gives PSNR 17.661 / LPIPS 0.388 versus 17.646 / 0.393 at 0.5 and 17.512 / 0.391 at 1.0 with depth init.
- Removing SDEdit (depth init only) drops average PSNR to 16.367 and LPIPS to 0.406 on 360-USID.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use AuraFusion360 instead of SPIn-NeRF | 360° scene inpainting · 360-USID Average | PSNR: 16.734 -> 17.661 | Helped |
| use AuraFusion360 instead of SPIn-NeRF | 360° scene inpainting · 360-USID Average | LPIPS: 0.464 -> 0.388 | Helped |
| add SDEdit (ablation) | AuraFusion360 · 360-USID Average | PSNR: 16.367 -> 17.661 | Helped |
| add SDEdit (ablation) | AuraFusion360 · 360-USID Average | LPIPS: 0.406 -> 0.388 | Helped |
| add depth initialization (ablation) | AuraFusion360 with SDEdit strength 0.85 · 360-USID Average | PSNR: 16.638 -> 17.661 | Helped |
| add depth initialization (ablation) | AuraFusion360 with SDEdit strength 0.85 · 360-USID Average | LPIPS: 0.456 -> 0.388 | Helped |
| set SDEdit strength to 0.5 instead of 0.85 (ablation) | AuraFusion360 · 360-USID Average | PSNR: 17.646 -> 17.661 | Hurt |
| set SDEdit strength to 0.5 instead of 0.85 (ablation) | AuraFusion360 · 360-USID Average | LPIPS: 0.393 -> 0.388 | Hurt |
| set SDEdit strength to 1.0 instead of 0.85 (ablation) | AuraFusion360 · 360-USID Average | PSNR: 17.512 -> 17.661 | Hurt |
| set SDEdit strength to 1.0 instead of 0.85 (ablation) | AuraFusion360 · 360-USID Average | LPIPS: 0.391 -> 0.388 | Hurt |
Surprising findings
- Gaussian Grouping and GScream score lower than the simple 2DGS+LaMa baseline on average in places, indicating that object-tracking-based unseen masks can hurt inpainting.
Limitations
- Computational efficiency is a stated concern for future work
- Limited to static scenes; dynamic scenes not addressed
- Some baselines (InNeRF360, [34]) could not be run for comparison
Experimental setup
Datasets: 360-USID Other-360 (NeRF, MipNeRF-360, Instruct-NeRF2NeRF scenes)
Benchmarks: 360-USID Other-360
Metrics: PSNR LPIPS
Baselines: SPIn-NeRF 2DGS + LaMa 2DGS + LeftRefill LeftRefill Gaussian Grouping GScream Infusion
Builds on: 2D Gaussian Splatting Marigold SDEdit LeftRefill Guided Depth Diffusion SAM2
Closest papers by meaning
- InstaInpaint: Instant 3D-Scene Inpainting with Masked Large Reconstruction Model (NeurIPS 2025 · similarity 0.87)
- IMFine: 3D Inpainting via Geometry-guided Multi-view Refinement (CVPR 2025 · similarity 0.86)
- 3D Gaussian Inpainting with Depth-Guided Cross-View Consistency (CVPR 2025 · similarity 0.85)
- EditSplat: Multi-View Fusion and Attention-Guided Optimization for View-Consistent 3D Scene Editing with 3D Gaussian Splatting (CVPR 2025 · similarity 0.85)
- SplatFlow: Multi-View Rectified Flow Model for 3D Gaussian Splatting Synthesis (CVPR 2025 · similarity 0.85)
- InNeRF360: Text-Guided 3D-Consistent Object Inpainting on 360-degree Neural Radiance Fields (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.