Research map / Image generation / Autoregressive and latent generation
SphereDiffusion: Spherical Geometry-Aware Distortion Resilient Diffusion Model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SphereDiffusion, a ControlNet-based framework for generating spherical panoramas from NFOV segmentation maps and text, using semantic encoding, deformable convolution, rotation-based training with SimSiam contrast, and rotated denoising to improve boundary continuity.
Problem
Controllable spherical panoramic image generation suffers from spherical distortion (object deformation breaking text-object correspondence in planar-pretrained models) and geometry issues such as boundary discontinuity, yielding low-quality outputs.
Approach
Replace RGB segmentation conditioning with CLIP label-embedding pixel maps (DRSE), add a deformable convolution block to the control encoder (DDaB), train with 3D-rotation data augmentation plus a SimSiam consistency loss in latent space, and rotate intermediate latents during denoising to connect panorama boundaries.
Key results
- On Structured3D, FID improves over ControlNet at all FOVs: 44.801→29.156 (30°), 41.917→26.262 (60°), 39.450→25.042 (90°), 35.690→24.147 (120°), about 35% relative on average.
- Ablation at 90° FOV: full model reaches FID 25.043 vs 39.450 baseline; Spherical SimSiam Contrastive Learning gives the largest drop (32.468→25.763).
- SGA Generation applied only at test time improves FID by about 0.7 without retraining.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SphereDiffusion instead of ControlNet | Controllable spherical panoramic image generation · Structured3D test, FOV 30° | FID: 44.801 -> 29.156 | Helped |
| Use SphereDiffusion instead of ControlNet | Controllable spherical panoramic image generation · Structured3D test, FOV 60° | FID: 41.917 -> 26.262 | Helped |
| Use SphereDiffusion instead of ControlNet | Controllable spherical panoramic image generation · Structured3D test, FOV 90° | FID: 39.450 -> 25.042 | Helped |
| Use SphereDiffusion instead of ControlNet | Controllable spherical panoramic image generation · Structured3D test, FOV 120° | FID: 35.690 -> 24.147 | Helped |
| Add DRSE to the baseline (ablation) | SphereDiffusion · Structured3D test, FOV 90° | FID: 39.450 -> 38.805 | Helped |
| Add DDaB to DRSE (ablation) | SphereDiffusion · Structured3D test, FOV 90° | FID: 38.805 -> 35.076 | Helped |
| Add Spherical Reprojection to DRSE + DDaB (ablation) | SphereDiffusion · Structured3D test, FOV 90° | FID: 35.076 -> 32.468 | Helped |
| Add Spherical SimSiam Contrastive Learning (ablation) | SphereDiffusion · Structured3D test, FOV 90° | FID: 32.468 -> 25.763 | Helped |
| Add SGA Generation during testing (ablation) | SphereDiffusion · Structured3D test, FOV 90° | FID: 25.763 -> 25.043 | Helped |
Limitations
- Evaluated only on a single dataset (Structured3D) with NFOV segmentation-map conditioning.
- Ablation gains from SGA Generation are small (about 0.7 FID).
- Rotation uses nearest-pixel interpolation, so rotated coordinates are approximate.
Experimental setup
Datasets: Structured3D
Benchmarks: Structured3D
Metrics: FID sFID IS
Baselines: ControlNet
Builds on: ControlNet Stable Diffusion Trans4PASS SimSiam CLIP
Closest papers by meaning
- Spherical Manifold Guided Diffusion Model for Panoramic Image Generation (CVPR 2025 · similarity 0.91)
- Spherical-Nested Diffusion Model for Panoramic Image Outpainting (ICML 2025 · similarity 0.90)
- DiffPano: Scalable and Consistent Text to Panorama Generation with Spherical Epipolar-Aware Diffusion (NeurIPS 2024 · similarity 0.86)
- Efficient Image Synthesis with Sphere Latent Encoder (NeurIPS 2026 · similarity 0.86)
- AdapToPASS: Ambiguity-aware Adaptive Spherical Transformer for Panoramic Semantic Segmentation (NeurIPS 2026 · similarity 0.86)
- Splatter-360: Generalizable 360 Gaussian Splatting for Wide-baseline Panoramic Images (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.