Research map / Image generation / Autoregressive and latent generation
HORIZON: High-Resolution Semantically Controlled Panorama Synthesis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HORIZON, a two-stage VQGAN-token transformer framework with Spherical Parallel Modeling (spherical relative embedding and spherical conditioning) for semantically controlled, high-resolution 360-degree panorama synthesis with image and text guidance.
Problem
Panorama synthesis suffers from spherical distortion, left-right edge discontinuity, lack of semantic user control, and slow generation at high resolution.
Approach
Models the 18 VQGAN view-token groups of an equirectangular panorama with a masked parallel decoder (MaskGIT-style) plus a second pass conditioned on a full spherical neighborhood, using spherical rotary embeddings that wrap around the longitude and CLIP image/text embeddings for semantic guidance.
Key results
- SPM(+SRE+SC) reaches FID 7.79 and spherical FID mean 20.97 on StreetLearn panorama generation, versus 36.33 FID for Text2light.
- Continuity score LRCS of 0.0020 for SPM(+SRE+SC), about 10x lower than Text2light's 0.0224.
- View extrapolation SSIM 0.542 and PSNR 15.49 for SPM, above ARM (0.521) and LPM (0.508).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add spherical conditioning (SC) to SPM with spherical relative embedding (SRE) (ablation) | SPM · StreetLearn Pittsburgh test | FID: 10.74 -> 7.79 | Helped |
| Add spherical conditioning (SC) to SPM with spherical relative embedding (SRE) (ablation) | SPM · StreetLearn Pittsburgh test | LRCS: 0.0726 -> 0.0020 | Helped |
| Replace autoregressive modeling with local parallel modeling (ablation) | Panorama generation · StreetLearn Pittsburgh test | FID: 25.36 -> 45.71 | Hurt |
| Replace local parallel modeling with spherical parallel modeling (ablation) | Panorama generation · StreetLearn Pittsburgh test | FID: 45.71 -> 7.79 | Helped |
| Use SPM with spherical relative embedding and spherical conditioning instead of ARM (ablation) | Panorama generation · StreetLearn Pittsburgh test | LRCS: 0.0283 -> 0.0020 | Helped |
| Use SPM rather than ARM for view extrapolation | HORIZON · StreetLearn view extrapolation | SSIM: 0.521 -> 0.542 | Helped |
| Use SPM rather than ARM for view extrapolation | HORIZON · StreetLearn view extrapolation | FID: 11.78 -> 5.53 | Helped |
| Use SPM rather than ARM on Matterport3D | HORIZON · Matterport3D | FID: 24.86 -> 8.26 | Helped |
| Use SPM rather than ARM on Matterport3D | HORIZON · Matterport3D | LRCS: 0.0279 -> 0.0021 | Helped |
| Use local parallel modeling rather than ARM (ablation) | HORIZON · StreetLearn generation, 2048x1024 | Img/s: 0.014 -> 0.396 | Helped |
| Use spherical parallel modeling rather than ARM (ablation) | HORIZON · StreetLearn generation, 2048x1024 | Img/s: 0.014 -> 0.195 | Helped |
| Use HORIZON rather than OmniDreamer | HORIZON · StreetLearn view extrapolation, 1024x512 | FID: 10.06 -> 6.93 | Helped |
Surprising findings
- Plain local parallel modeling (LPM) is about 64x faster but markedly worse in FID (45.71) and continuity (LRCS 0.2032) than autoregressive modeling.
- Applying spherical conditioning directly on LPM did not yield strong results; the coarse-then-refine two-pass design was needed.
Limitations
- Artifacts may stem from the limited capacity of the VQGAN encoder/decoder.
- Comparisons with prior panorama methods are limited because their datasets are not public; several baselines were retrained on StreetLearn.
- Inference time of SPM is roughly doubled by the second refinement pass.
Experimental setup
Datasets: StreetLearn (Pittsburgh) Matterport3D
Benchmarks: StreetLearn Matterport3D
Metrics: FID spherical FID LRCS SSIM PSNR
Baselines: Text2light ARM LPM COCO-GAN InfinityGAN OmniDreamer
Builds on: VQGAN MaskGIT RoPE CLIP
Closest papers by meaning
- CubeDiff: Repurposing Diffusion-Based Image Models for Panorama Generation (ICLR 2025 · similarity 0.88)
- DiffPano: Scalable and Consistent Text to Panorama Generation with Spherical Epipolar-Aware Diffusion (NeurIPS 2024 · similarity 0.87)
- PanSplat: 4K Panorama Synthesis with Feed-Forward Gaussian Splatting (CVPR 2025 · similarity 0.87)
- ViewPoint: Panoramic Video Generation with Pretrained Diffusion Models (NeurIPS 2025 · similarity 0.87)
- SceneCraft: Layout-Guided 3D Scene Generation (NeurIPS 2024 · similarity 0.86)
- 4K4DGen: Panoramic 4D Generation at 4K Resolution (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.