Research map / Image generation / Text-to-image diffusion
Conditional Panoramic Image Generation via Masked Autoregressive Modeling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PAR, a masked autoregressive (MAR) framework built on NOVA that unifies text-to-panorama and panorama outpainting/editing, with dual-space circular padding and a cyclic translation consistency loss for ERP panoramas.
Problem
Conditional panoramic image generation: diffusion models violate the i.i.d. Gaussian noise assumption on equirectangular projection (ERP) panoramas, and existing methods separate text-to-panorama and outpainting into task-specific pipelines and data.
Approach
Use masked autoregressive modeling with continuous tokens and a diffusion MLP head (no i.i.d. noise on ERP pixels), treat outpainting as partially observed token generation so both tasks share one likelihood objective, and adapt pretrained perspective-trained models to ERP via circular padding in pixel and latent space plus a translation-equivariance consistency loss.
Key results
- PAR-1.4B reaches 37.37 FID on Matterport3D text-to-panorama, versus 45.21 for PanFusion (1.9B diffusion).
- PAR-0.3B reaches 3.39 FAED and 0.58 DS, outperforming PanFusion's FAED of 5.12 and beating PanoLlama and Text2Light on FAED and DS.
- On Matterport3D outpainting, PAR w/ prompt achieves 32.68 FID and 12.20 FID-h, versus 52.59 and 35.18 for 2S-ODIS.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PAR-0.3B instead of PanFusion | Matterport3D text-to-panorama · Matterport3D | FID: 45.21 -> 41.15 | Helped |
| Use PAR-0.3B instead of PanFusion | Matterport3D text-to-panorama · Matterport3D | CLIP Score: 31.07 -> 30.21 | Hurt |
| Use PAR-1.4B instead of PanFusion | Matterport3D text-to-panorama · Matterport3D | FID: 45.21 -> 37.37 | Helped |
| Add a textual prompt for outpainting (ablation) | PAR · Matterport3D panorama outpainting | FID: 41.63 -> 32.68 | Helped |
| Add a textual prompt for outpainting (ablation) | PAR · Matterport3D panorama outpainting | FID-h: 25.97 -> 12.20 | Helped |
| Add cyclic translation consistency loss (ablation) | PAR · Matterport3D text-to-panorama | FID: 39.55 -> 37.37 | Helped |
| Add cyclic translation consistency loss (ablation) | PAR · Matterport3D text-to-panorama | CLIP Score: 30.25 -> 30.41 | Helped |
| Add cyclic translation consistency loss (ablation) | PAR · Matterport3D text-to-panorama | DS: 0.57 -> 0.58 | Hurt |
| Vary denoising steps (ablation) | PAR-0.3B · Matterport3D text-to-panorama | FID: 41.57, 40.19, 41.98, 43.77 | Mixed |
| Increase CFG coefficient from 3 to 10 (ablation) | PAR-1.4B · Matterport3D text-to-panorama | FID: 40.04 -> 39.76 | Helped |
| Increase AR steps (ablation) | PAR-0.3B · inference | inference speed (sec/image): 3.02, 5.49, 10.03 | Hurt |
| Apply pre-padding (ablation) | PAR · panorama generation | DS | Helped |
Surprising findings
- Consistency loss gives only a small FID gain (39.55 to 37.37) while CLIP score and DS are essentially unchanged or slightly worse.
- PAR-0.3B is reported with a lower FID (41.15) than PanFusion (45.21) and substantially lower inference time (10.03 vs 28.91 sec/image).
Limitations
- Gap remains between generated images and real panoramas in details and textures, attributed to scarce panoramic training data.
- Failures on fine details of small objects such as tables and sofas.
- Relies on panoramic data to adapt a perspective-pretrained model, which may be less efficient than other approaches.
Experimental setup
Datasets: Matterport3D SUN360 Structured3D
Benchmarks: Matterport3D SUN360 Structured3D
Metrics: FID FAED CLIP Score Discontinuity Score FID-h
Baselines: PanFusion Text2Light PanoLlama AOG-Net 2S-ODIS Diffusion360 PanoDiff DiffPano UniPano StitchDiffusion
Builds on: masked autoregressive modeling (MAR) NOVA classifier-free guidance
Closest papers by meaning
- DiffPano: Scalable and Consistent Text to Panorama Generation with Spherical Epipolar-Aware Diffusion (NeurIPS 2024 · similarity 0.89)
- Taming Stable Diffusion for Text to 360 Panorama Image Generation (CVPR 2024 · similarity 0.88)
- CubeDiff: Repurposing Diffusion-Based Image Models for Panorama Generation (ICLR 2025 · similarity 0.87)
- PanoWan: Lifting Diffusion Video Generation Models to 360$^\circ$ with Latitude/Longitude-aware Mechanisms (NeurIPS 2025 · similarity 0.87)
- EditAR: Unified Conditional Generation with Autoregressive Models (CVPR 2025 · similarity 0.87)
- Spherical-Nested Diffusion Model for Panoramic Image Outpainting (ICML 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.