Research map / Image generation / Autoregressive and latent generation
Spherical Manifold Guided Diffusion Model for Panoramic Image Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SMGD, a text-conditioned panoramic image diffusion model built on a spherical manifold convolution (SMConv) via exponential mapping, plus a cubemap-based grouped FID evaluation.
Problem
Text-conditioned panoramic (ERP) image generation is hard because equirectangular projection introduces spherical distortion, breaks left-right continuity, and yields sparse pixels near the poles; existing generators handle these poorly and are slow.
Approach
Performs convolution directly on the S2 sphere by projecting tangent-space kernels onto the sphere with exponential mapping (SMConv), uses it in a U-Net denoiser with a WS-PSNR-weighted spherical loss, and evaluates quality with FID computed separately on equatorial and polar cubemap faces.
Key results
- SMGD reaches FID 9.85 and FAED 0.22 versus 16.15 and 0.35 for the second-best PanFusion, with CLIP score 0.189.
- SMGD samples in 2.69 s per image versus 38.51 s for PanFusion on an NVIDIA 4090, with 1.4B parameters versus PanFusion's 2.3B.
- Ablation shows the spherical loss, SMD blocks, SME blocks and SPB each reduce FID, with the full model going from 23.11 to 9.85.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SMGD instead of PanFusion | Text-conditioned panoramic image generation · Matterport3D | FIDavg: 14.34 -> 9.70 | Helped |
| Use SMGD instead of PanFusion | Text-conditioned panoramic image generation · Matterport3D | CS: 0.182 -> 0.189 | Helped |
| Use SMGD instead of PanFusion | Text-conditioned panoramic image generation · Matterport3D | sampling time (s): 38.51 ± 11.57 -> 2.69 ± 0.39 | Helped |
| Use SMGD instead of PanFusion | Text-conditioned panoramic image generation · Matterport3D equatorial regions | FIDequ: 14.81 -> 7.88 | Helped |
| Use SMGD instead of PanFusion | Text-conditioned panoramic image generation · Matterport3D polar regions | FIDpole: 31.99 -> 21.05 | Helped |
| Add spherical loss (LSMSE) (ablation) | SMGD · Matterport3D | FIDavg: 20.57 -> 17.50 | Helped |
| Add SMD blocks (ablation) | SMGD · Matterport3D | FIDavg: 17.50 -> 12.87 | Helped |
| Add SME blocks (ablation) | SMGD · Matterport3D | FIDavg: 12.87 -> 11.42 | Helped |
| Add SPB modules (ablation) | SMGD · Matterport3D | FIDavg: 11.42 -> 9.70 | Helped |
Surprising findings
- Standard FID on random or center crops of ERP panoramas ranks MVDiffusion and SD2+LoRA well on FIDavg yet they have very high FIDpole, indicating crop-based FID misses polar distortion.
Limitations
- Spherical convolution with bilinear interpolation consumes much more GPU memory at high resolution, so SMConv is used only in deeper layers.
- Evaluated only on Matterport3D-derived captions and Structured3D, with a relatively small dataset of about 10K images.
Experimental setup
Datasets: Matterport3D Structured3D
Benchmarks: Matterport3D (10K panoramas) Structured3D
Metrics: FID FIDequ FIDpole FIDavg OmniFID FAED CLIP score inference time
Baselines: Multi-Diffusion SD2+LoRA Text2Light MVDiffusion PanFusion LDM+SphereNet
Builds on: latent diffusion model SphereNet VQGAN classifier-free guidance DDIM
Closest papers by meaning
- SphereDiffusion: Spherical Geometry-Aware Distortion Resilient Diffusion Model (AAAI 2024 · similarity 0.91)
- Spherical-Nested Diffusion Model for Panoramic Image Outpainting (ICML 2025 · similarity 0.90)
- DiffPano: Scalable and Consistent Text to Panorama Generation with Spherical Epipolar-Aware Diffusion (NeurIPS 2024 · similarity 0.85)
- Splatter-360: Generalizable 360 Gaussian Splatting for Wide-baseline Panoramic Images (CVPR 2025 · similarity 0.85)
- Conditional Panoramic Image Generation via Masked Autoregressive Modeling (NeurIPS 2025 · similarity 0.85)
- ViewPoint: Panoramic Video Generation with Pretrained Diffusion Models (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.