Research map / Video / Video generation
Imagine360: Immersive 360 Video Generation from Perspective Anchor
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Imagine360 lifts a perspective video into a 360° equirectangular video using a dual-branch (panorama + perspective) AnimateDiff-based denoiser with antipodal cross-domain attention and elevation-aware mask designs.
Problem
Generating 360° equirectangular videos from ordinary perspective anchor videos, where the unknown canvas is large and the video masks change shape and location with camera elevation across frames.
Approach
A dual-branch panorama/perspective video denoiser bidirectionally coupled through spherical cross-domain attention, extended with an antipodal mask to model reversed motion across hemispheres and elevation-aware sampling and estimation to handle changing masks.
Key results
- Imagine360 achieves imaging quality 0.7487 and VQA 0.8672 on VBench/Q-Align, beating Follow-Your-Canvas (0.6978, 0.8444) and 360DVD (0.5537, 0.5867) on 100 test cases.
- In human evaluation with 26 experts, Imagine360 scores 3.68 graphics quality, 3.73 structure plausibility and 3.39 temporal coherence, the highest of all compared methods.
- On panorama image outpainting, Imagine360 reports Intra-Style 0.99e-3, CLIP 29.12 and IQA 0.78, best among Diffusion360, PanoDiffusion and SIG-SS.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Imagine360 instead of Follow-Your-Canvas | 360° video generation · 360° video generation | VQA: 0.8444 -> 0.8672 | Helped |
| Use Imagine360 instead of Follow-Your-Canvas | 360° video generation · 360° video generation | Graphics Quality: 2.7692 -> 3.6827 | Helped |
| Use Imagine360 for panorama image outpainting instead of Diffusion360 | Panorama image outpainting · Panorama image outpainting | CLIP: 27.49 -> 29.12 | Helped |
| Add a perspective branch to the panorama-only Imagine360 model (ablation) | Imagine360 · 360° video generation | VQA: 0.6168 -> 0.8672 | Helped |
| Add a panorama branch to the perspective-only Imagine360 model (ablation) | Imagine360 · 360° video generation | VQA: 0.8435 -> 0.8672 | Helped |
| Add the antipodal mask (ablation) | Imagine360 · 360° video generation | VQA: 0.8580 -> 0.8672 | Helped |
| Add elevation-aware designs (ablation) | Imagine360 · 360° video generation | VQA: 0.7663 -> 0.8672 | Helped |
| Add perspective spatial LoRA (ablation) | Imagine360 · 360° video generation | VQA: 0.8584 -> 0.8672 | Helped |
| Add panorama spatial LoRA (ablation) | Imagine360 · 360° video generation | VQA: 0.8553 -> 0.8672 | Helped |
| Fine-tune the panorama motion module instead of using motion LoRA (ablation) | Imagine360 · 360° video generation | VQA: 0.8385 -> 0.8672 | Helped |
| Train 360DVD on Imagine360's extended panorama video data (ablation) | 360DVD · 360° video generation | VQA: 0.5867 -> 0.7573 | Helped |
| Smooth the estimated elevation sequence (ablation) | Imagine360 · 360° video generation | VQA: 0.8443 -> 0.8672 | Helped |
Surprising findings
- Motion LoRA on the panorama motion module degraded quality relative to fully fine-tuning the motion module, which the authors attribute to a larger temporal domain gap.
Limitations
- Relies on off-the-shelf PerspectiveFields elevation estimation, which can cause artifacts from inaccurate elevation estimates.
- Evaluation on only 100 test cases with a small set of baselines.
- Elevation predictor designed for images, producing temporally inconsistent pitch without smoothing.
Experimental setup
Datasets: WEB360 YouTube 360-degree videos (self-collected)
Benchmarks: VBench Q-Align VQA human evaluation panorama image outpainting comparison
Metrics: imaging quality aesthetic quality motion smoothness subject consistency VQA graphics quality structure plausibility temporal coherence Intra-Style CLIP score
Baselines: AnimateDiff+LatentLab360 Follow-Your-Canvas 360DVD Diffusion360 PanoDiffusion SIG-SS
Builds on: AnimateDiff PanFusion Follow-Your-Canvas IP-Adapter 360DVD
Closest papers by meaning
- ViewPoint: Panoramic Video Generation with Pretrained Diffusion Models (NeurIPS 2025 · similarity 0.87)
- MVSplat360: Feed-Forward 360 Scene Synthesis from Sparse Views (NeurIPS 2024 · similarity 0.86)
- 360DVD: Controllable Panorama Video Generation with 360-Degree Video Diffusion Model (CVPR 2024 · similarity 0.86)
- PanoWan: Lifting Diffusion Video Generation Models to 360$^\circ$ with Latitude/Longitude-aware Mechanisms (NeurIPS 2025 · similarity 0.86)
- 4K4DGen: Panoramic 4D Generation at 4K Resolution (ICLR 2025 · similarity 0.86)
- From an Image to a Scene: Learning to Imagine the World from a Million 360° Videos (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.