Research map / Video / Video generation
360DVD: Controllable Panorama Video Generation with 360-Degree Video Diffusion Model
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes 360DVD, a lightweight 360-Adapter plus 360 enhancement techniques (latitude-aware loss, latent rotation, circular padding) that adapt AnimateDiff/Stable Diffusion for controllable 360-degree panorama video generation, trained on a new WEB360 captioned panoramic video dataset.
Problem
Text-to-video diffusion models trained on standard videos produce poor equirectangular 360-degree panorama videos, due to differences in content distribution, motion patterns, and left-right continuity; no captioned panoramic video dataset existed.
Approach
Freeze a pre-trained T2V model (SD v1.5 + AnimateDiff motion module) and train a lightweight pseudo-3D 360-Adapter that injects optical-flow motion features into the U-Net encoder, combined with panorama-specific enhancements (latitude-weighted loss, latent rotation at inference, late-stage circular padding) and a GPT-fused caption dataset.
Key results
- In a 31-participant user study, 360DVD was preferred on end continuity (71.8%), content distribution (67.0%), and motion pattern (74.6%) over AnimateDiff and the other variants.
- 360DVD was preferred for frame consistency at 64.5% and graphics quality at 51.6%.
- Generates 512x1024 ERP panorama videos across personalized SD models (Realistic Vision, Lyriel, ToonYou, RCNZ Cartoon) with motion conditioning from optical flow.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add 360 Enhancement Techniques (latent rotation, circular padding, latitude-aware loss) to AnimateDiff+LoRA (ablation) | AnimateDiff with panoramic LoRA · User study (31 participants) | user preference % across criteria: Graphics 14.1→23.0, End Continuity 6.0→16.9, Content 12.1→16.1, Motion 6.5→14.5 | Helped |
| Full 360DVD (360-Adapter plus 360 Enhancement Techniques, WEB360 training) versus baselines | AnimateDiff-based panorama generation · User study (31 participants) | user preference % across criteria: Highest preference on all five criteria, e.g. Motion Pattern 74.6% vs 4.4% for AnimateDiff | Helped |
| Ablation of 360 Text Fusion, pseudo-3D layer, and latitude-aware loss (ablation) | 360DVD · Qualitative ablation (Fig. 8) | qualitative visual quality | Helped |
| Circular padding applied only in late half of denoising steps | 360DVD inference · Method design | not quantified | No clear effect |
Limitations
- Output resolution limited to 512x1024 by SD constraints; relies on super-resolution for higher resolution.
- Evaluation is mainly qualitative plus a user study with 31 participants on 8 sets; no standard quantitative video metrics reported.
- Dataset is small (about 2,000 clips) and built partly from BLIP/ChatGPT captions.
Experimental setup
Datasets: WEB360 ODV360
Benchmarks: WEB360 (qualitative and user study)
Metrics: user preference percentage over graphics quality, frame cons
Baselines: AnimateDiff AnimateDiff+LoRA (LatentLabs360) AnimateDiff+360ET
Builds on: AnimateDiff Stable Diffusion PanoDiff PanoFlow
Closest papers by meaning
- PanoDiT: Panoramic Videos Generation with Diffusion Transformer (AAAI 2025 · similarity 0.91)
- ViewPoint: Panoramic Video Generation with Pretrained Diffusion Models (NeurIPS 2025 · similarity 0.90)
- PanoWan: Lifting Diffusion Video Generation Models to 360$^\circ$ with Latitude/Longitude-aware Mechanisms (NeurIPS 2025 · similarity 0.88)
- DiffPano: Scalable and Consistent Text to Panorama Generation with Spherical Epipolar-Aware Diffusion (NeurIPS 2024 · similarity 0.87)
- DynamicScaler: Seamless and Scalable Video Generation for Panoramic Scenes (CVPR 2025 · similarity 0.86)
- CubeDiff: Repurposing Diffusion-Based Image Models for Panorama Generation (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.