Research map / Video / Video generation
ViewPoint: Panoramic Video Generation with Pretrained Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Generates 360-degree panoramic videos from text and/or perspective video by fine-tuning Wan2.1 on a novel ViewPoint map (pseudo-perspective stitched cubemap) with Pano-Perspective attention.
Problem
Panoramic video generation from perspective inputs or text is hard because ERP distorts content and falls outside diffusion-model training distributions, while cubemaps have discontinuous face borders.
Approach
A ViewPoint map rearranges cubemap faces into overlapping pseudo-perspective panels to keep continuity while preserving local distortion-free views; Pano-attention blocks model global context over the whole map and Perspective-attention blocks model each panel, reusing pretrained perspective video priors.
Key results
- On ODV360 VBench evaluation, ViewPoint achieves subject consistency 0.8793, imaging quality 0.5927, motion smoothness 0.9800, dynamic degree 0.9083, best among compared methods.
- Ablation: ERP baseline reaches dynamic degree 0.8840 and imaging quality 0.5396; CP-ICLoRA gets highest imaging quality 0.6018 but has visible cube-face discontinuities.
- User study with 50 valid questionnaires shows ViewPoint receives the most votes across all four dimensions.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use ViewPoint rather than the strongest baseline | panoramic video generation · ODV360 | subject consistency | Helped |
| use ViewPoint rather than the strongest baseline | panoramic video generation · ODV360 | imaging quality | Helped |
| use ViewPoint rather than the strongest baseline | panoramic video generation · ODV360 | motion smoothness | Helped |
| use ViewPoint rather than the strongest baseline | panoramic video generation · ODV360 | dynamic degree | Helped |
| use ERP representation instead of ViewPoint Map (ablation) | ViewPoint · ODV360 | subject consistency: 0.8793 -> 0.8702 | Hurt |
| use cubemap with independent positional encoding per face instead of ViewPoint Map (ablation) | ViewPoint · ODV360 | imaging quality: 0.5927 -> 0.5104 | Hurt |
| arrange cube faces in a 2-row by 3-column raster layout instead of ViewPoint Map (ablation) | ViewPoint · ODV360 | imaging quality: 0.5927; 0.6018 | Mixed |
| remove Perspective-Attention (ablation) | ViewPoint · ODV360 | subject consistency: 0.8793 -> 0.8663 | Hurt |
| remove Perspective-Attention (ablation) | ViewPoint · ODV360 | imaging quality: 0.5927 -> 0.5392 | Hurt |
| remove Perspective-Attention (ablation) | ViewPoint · ODV360 | motion smoothness: 0.9800 -> 0.9798 | Hurt |
| remove Perspective-Attention (ablation) | ViewPoint · ODV360 | dynamic degree: 0.9083 -> 0.8827 | Hurt |
Surprising findings
- CP-ICLoRA scores highest on imaging quality yet shows noticeable spatial discontinuities at cube-face boundaries, so per-face quality does not reflect panoramic quality.
- 360DVD achieves the second-best subject consistency score but the lowest dynamic degree, as its videos are nearly static.
Limitations
- Evaluation relies on perspective VBench metrics applied to projected views and a small user study (50 questionnaires), not panoramic-specific metrics.
- Training data is limited to few panorama datasets with captions partly generated by Qwen-VL.
Experimental setup
Datasets: Flickr360 WEB360 ODV360 360+x
Benchmarks: ODV360 (VBench-based evaluation on projected perspective views)
Metrics: subject consistency imaging quality motion smoothness dynamic degree user study ratings (spatial continuity, temporal quality, ae
Baselines: 360DVD Imagine360 Follow-Your-Canvas ERP fine-tuning CP-ICLoRA CP-4DRoPE
Builds on: Wan2.1 flow matching in-context generation
Closest papers by meaning
- PanoWorld: Geometry-Consistent Panoramic Video World Modeling (NeurIPS 2026 · similarity 0.92)
- PanoWan: Lifting Diffusion Video Generation Models to 360$^\circ$ with Latitude/Longitude-aware Mechanisms (NeurIPS 2025 · similarity 0.91)
- 360DVD: Controllable Panorama Video Generation with 360-Degree Video Diffusion Model (CVPR 2024 · similarity 0.90)
- 4K4DGen: Panoramic 4D Generation at 4K Resolution (ICLR 2025 · similarity 0.90)
- DynamicScaler: Seamless and Scalable Video Generation for Panoramic Scenes (CVPR 2025 · similarity 0.89)
- PanoDiT: Panoramic Videos Generation with Diffusion Transformer (AAAI 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.