Research map / Image generation / Text-to-image diffusion
Autoregressive Omni-Aware Outpainting for Open-Vocabulary 360-Degree Image Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AOG-Net, an autoregressive diffusion-based outpainting method that generates 360-degree images from NFoV images and open-vocabulary text using a global-local cross-attention conditioning mechanism.
Problem
Generating high-fidelity 360-degree images from NFoV images and open-vocabulary text while keeping fine details and text consistency and preserving omni-directional continuity.
Approach
Outpaints an incomplete equirectangular 360 image step by step along longitude using NFoV gnomonic views, with a stable diffusion backbone conditioned by a global stream (omni-visual CLIP features cross-attending to text) and a local stream (NFoV and omni-geometry features cross-attending to omni-visual features).
Key results
- Indoor FID 38.60 / LPIPS 0.37 in Table 1 vs OmniDreamer FID 46.15 and ImmerseGAN FID 42.78 (text in paper gives 35.6 FID, inconsistent with table).
- Outdoor FID 18.4 and LPIPS 0.36 vs OmniDreamer FID 24.5 / LPIPS 0.41.
- Against Text2Light, SC improves from 0.45 to 0.53 outdoors and from 0.33 to 0.36 indoors, with IS 4.2 vs 3.9 outdoors and 5.1 vs 4.5 indoors.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use AOG-Net instead of OmniDreamer | NFoV-guided 360-degree generation · LAVAL indoor HDR | FID: 46.15 -> 38.60 | Helped |
| use AOG-Net instead of OmniDreamer | NFoV-guided 360-degree generation · LAVAL outdoor HDR | FID: 24.5 -> 18.4 | Helped |
| use AOG-Net instead of Text2Light | text-guided 360-degree generation · LAVAL outdoor HDR | SC: 0.45 -> 0.53 | Helped |
| use AOG-Net instead of Text2Light | text-guided 360-degree generation · LAVAL indoor HDR | IS: 4.5 -> 5.1 | Helped |
| add global condition (ablation) | AOG-Net · LAVAL indoor HDR | FID: 40.08 -> 35.6 | Helped |
| add global condition (ablation) | AOG-Net · LAVAL indoor HDR | SC: 0.70 -> 0.72 | Helped |
| add local condition (ablation) | AOG-Net · LAVAL indoor HDR | FID: 47.46 -> 35.6 | Helped |
| add local condition (ablation) | AOG-Net · LAVAL indoor HDR | SC: 0.71 -> 0.72 | Helped |
| add geometry condition (ablation) | AOG-Net · LAVAL indoor HDR | FID: 37.2 -> 35.6 | Helped |
| add geometry condition (ablation) | AOG-Net · LAVAL indoor HDR | SC: 0.72 -> 0.72 | No clear effect |
| use global-local conditioning instead of only the Stable Diffusion backbone (ablation) | AOG-Net · LAVAL indoor HDR | FID: 67.4 -> 35.6 | Helped |
| use global-local conditioning instead of only the Stable Diffusion backbone (ablation) | AOG-Net · LAVAL indoor HDR | SC: 0.61 -> 0.72 | Helped |
Surprising findings
- Removing the geometry condition gives a slightly better LPIPS (0.36) than the full model, though FID and visual quality are worse.
Limitations
- Inherits biases of the pretrained diffusion backbone.
- Diffusion inference time is long, limiting real-time use.
- Outdoor evaluation is limited to a small test split of 40 images and few baselines report outdoor results.
Experimental setup
Datasets: LAVAL indoor HDR dataset LAVAL outdoor HDR dataset
Benchmarks: LAVAL indoor HDR LAVAL outdoor HDR
Metrics: FID LPIPS semantic consistency (SC) Inception Score (IS)
Baselines: OmniDreamer Text2Light ImmerseGAN EnvMapNet SIG-SS
Builds on: Stable Diffusion OmniDreamer T2I-Adapter OpenCLIP DPM-Solver++
Closest papers by meaning
- OmniGen-AR: AutoRegressive Any-to-Image Generation (NeurIPS 2025 · similarity 0.87)
- Continuous-Multiple Image Outpainting in One-Step via Positional Query and A Diffusion-based Approach (ICLR 2024 · similarity 0.86)
- Conditional Panoramic Image Generation via Masked Autoregressive Modeling (NeurIPS 2025 · similarity 0.86)
- MVSplat360: Feed-Forward 360 Scene Synthesis from Sparse Views (NeurIPS 2024 · similarity 0.86)
- Visual Autoregressive Modeling: Scalable Image Generation via Next-Scale Prediction (NeurIPS 2024 · similarity 0.85)
- RandAR: Decoder-only Autoregressive Visual Generation in Random Orders (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.