Research map / Image generation / Image restoration
Spherical-Nested Diffusion Model for Panoramic Image Outpainting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SpND, a diffusion-based panoramic image outpainting method that injects sphere geometry via i.i.d. spherical noise projected to ERP and a spherical deformable convolution (SDC) layer on top of a frozen pre-trained Stable Diffusion/ControlNet.
Problem
Panoramic image outpainting in ERP format suffers from spherical distortion and boundary continuity; existing methods only enforce sphere properties via soft regularisation losses or extra cues, failing to fully exploit spherical geometry.
Approach
Embeds the sphere nature directly into the diffusion architecture: noise is sampled i.i.d. on the 3D sphere and projected to ERP as a structural prior, and a spherical deformable convolution uses a spherical (gnomonic-projected) grid as the initial sampling grid with constrained learnable offsets, plus a circular mask encoder for edge consistency.
Key results
- On Matterport3D, SpND reaches FID 9.08 and FIDhori 10.64 versus 39.43 / 17.34 for PanoDiffusion; SpNDprompt reaches FID 6.67.
- On Structured3D, SpND achieves FID 7.37 vs 9.80 for PanoDiffusion; SpNDprompt achieves FID 4.60.
- Ablation: removing the structural prior raises Matterport3D FID from 9.08 to 9.93, and replacing SDC with SphereNet raises it to 11.78.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SpND instead of PanoDiff | Panoramic image outpainting · Matterport3D | FID: 13.45 -> 9.08 | Helped |
| Use SpND instead of PanoDiffusion | Panoramic image outpainting · Structured3D | FID: 9.80 -> 7.37 | Helped |
| Use specific text prompts instead of the general prompt “Outpainting” | SpND · Matterport3D | FID: 9.08 -> 6.67 | Helped |
| Add ERP structural prior (ablation) | SpND · Matterport3D | FID: 9.93 -> 9.08 | Helped |
| Add the spherical grid to the SDC layer (ablation) | SpND · Matterport3D | FID: 11.64 -> 9.08 | Helped |
| Add learnable offsets to the SDC layer (ablation) | SpND · Matterport3D | FID: 12.76 -> 9.08 | Helped |
| Use SpND's structural prior and spherical nested blocks instead of Repaint (ablation) | Panoramic image outpainting · Matterport3D | FID: 10.10 -> 9.08 | Helped |
| Use deformable convolution and SDC instead of spherical convolution (ablation) | SpND · Matterport3D | FID: 11.78 -> 9.08 | Helped |
| Set spherical-noise sampling density ζ to 15 instead of 30 (ablation) | SpND · Matterport3D | FID: 9.08 -> 12.74 | Hurt |
| Set spherical-noise sampling density ζ to 45 instead of 30 (ablation) | SpND · Matterport3D | FID: 9.08 -> 8.84 | Helped |
| Replace spherical noise with i.i.d. noise (ζ = ∞) (ablation) | SpND · Matterport3D | FID: 9.08 -> 9.93 | Hurt |
Surprising findings
- Removing the learnable offsets from SDC (spherical grid only) degrades results substantially, indicating the spherical grid alone is insufficient.
Limitations
- Repaint in latent space causes slight distortion between input and output near mask boundaries.
- Diversity slightly reduced when integrating the sphere nature (Repaint variant has higher recall).
Experimental setup
Datasets: Matterport3D Structured3D
Benchmarks: Matterport3D Structured3D
Metrics: FID FIDhori precision recall average of precision and recall
Baselines: SIG-SS TFill OmniDreamer PanoDiff AOG-Net PanoDiffusion
Builds on: ControlNet Stable Diffusion deformable convolution RePaint SphereNet
Closest papers by meaning
- SphereDiffusion: Spherical Geometry-Aware Distortion Resilient Diffusion Model (AAAI 2024 · similarity 0.90)
- Spherical Manifold Guided Diffusion Model for Panoramic Image Generation (CVPR 2025 · similarity 0.90)
- PanoDiffusion: 360-degree Panorama Outpainting via Diffusion (ICLR 2024 · similarity 0.87)
- DiffPano: Scalable and Consistent Text to Panorama Generation with Spherical Epipolar-Aware Diffusion (NeurIPS 2024 · similarity 0.87)
- Conditional Panoramic Image Generation via Masked Autoregressive Modeling (NeurIPS 2025 · similarity 0.86)
- In-N-Out: Lifting 2D Diffusion Prior for 3D Object Removal via Tuning-Free Latents Alignment (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.