Research map / Video / Video generation
PanoWan: Lifting Diffusion Video Generation Models to 360$^\circ$ with Latitude/Longitude-aware Mechanisms
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Lifts a pretrained text-to-video model (Wan 2.1 1.3B) to text-to-panoramic-video generation via LoRA plus latitude-aware sampling, rotated semantic denoising, and padded pixel-wise decoding, trained on a new 13K-clip panoramic dataset PanoVid.
Problem
Panoramic (360°) video generation struggles to reuse pretrained conventional text-to-video priors due to scarce paired panoramic data and the spatial gap of equirectangular projection (latitudinal distortion and longitude seam discontinuities).
Approach
Minimal modules adapt a conventional video DiT to ERP panoramas: latitude-remapped initial noise to counter polar stretching, rolling the latent grid during denoising to spread seam error, and circular-padded VAE decoding to make seams pixel-consistent, with only LoRA weights trained.
Key results
- Full PanoWan reaches FVD 1281.21 and VideoCLIP-XL 21.86, versus 1750.36 / 20.27 for 360DVD and 2146.04 / 21.13 for DynamicScaler.
- End continuity improves to 0.0270 with all modules, compared with 0.0327 without rotated semantic denoising and 0.0294 without padded pixel-wise decoding.
- Only 21.9M LoRA parameters (about 1.6% of the base model) are trained for 18 hours on 8 H100 GPUs.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add latitude-aware sampling (LAS) (ablation) | PanoWan · Panoramic video generation | FVD: 1520.69 -> 1281.21 | Helped |
| add latitude-aware sampling (LAS) (ablation) | PanoWan · Panoramic video generation | VideoCLIP-XL: 21.20 -> 21.86 | Helped |
| add rotated semantic denoising (RSD) (ablation) | PanoWan · Panoramic video generation | End Continuity: 0.0327 -> 0.0270 | Helped |
| add padded pixel-wise decoding (PPD) (ablation) | PanoWan · Panoramic video generation | End Continuity: 0.0294 -> 0.0270 | Helped |
Limitations
- Content forgetting over long videos due to limited temporal memory, inherited from the base model.
- Occasional failures in object consistency and common-sense structure (e.g., pet dog features, book structure), inherited from the backbone.
Experimental setup
Datasets: PanoVid 360-1M 360+x Imagine360 WEB360 Panonut360 Miraikan 360-degree Video Dataset
Benchmarks: PanoVid test comparison (360DVD and DynamicScaler baselines)
Metrics: FVD VideoCLIP-XL image quality end continuity motion pattern (user preference) scene richness (user preference)
Baselines: 360DVD DynamicScaler
Builds on: Wan 2.1 GEN3C LoRA flow matching OmniFID
Closest papers by meaning
- ViewPoint: Panoramic Video Generation with Pretrained Diffusion Models (NeurIPS 2025 · similarity 0.91)
- PanoWorld: Geometry-Consistent Panoramic Video World Modeling (NeurIPS 2026 · similarity 0.90)
- PanoDiT: Panoramic Videos Generation with Diffusion Transformer (AAAI 2025 · similarity 0.90)
- DiffPano: Scalable and Consistent Text to Panorama Generation with Spherical Epipolar-Aware Diffusion (NeurIPS 2024 · similarity 0.89)
- DynamicScaler: Seamless and Scalable Video Generation for Panoramic Scenes (CVPR 2025 · similarity 0.88)
- 360DVD: Controllable Panorama Video Generation with 360-Degree Video Diffusion Model (CVPR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.