Research map / Video / Video generation
4K4DGen: Panoramic 4D Generation at 4K Resolution
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Turns a single 4K equirectangular panorama into a dynamic 360° 4D scene by animating it with a spherical-latent Panoramic Denoiser built on a perspective I2V model and lifting it to dynamic 3D Gaussians with spatial-temporal geometry alignment.
Problem
Generating immersive, free-viewpoint 360° dynamic 4D environments at high (4K) resolution from a single panorama, where 4D panoramic training data is scarce and 2D diffusion models are trained on narrow-FoV perspective images.
Approach
Denoise spherical panoramic latents by projecting them into overlapping perspective views, denoising each with a pretrained perspective I2V denoiser (Animate-Anything/SVD), and fusing back to the sphere at every step; then lift the panoramic video to time-dependent 3D Gaussians using depth alignment across perspective views plus temporal, semantic (DINO) and geometric regularization.
Key results
- 4K4DGen achieves FID 16.59 and KID 0.2 (x10^-2) versus 57.05/1.4 for 3D-Cinemagraphy (zoom-in), and is chosen as best visual quality by 81% of user-study votes.
- In the animating-phase ablation, the spherical denoiser reaches Q-Align VQ 0.85 and motion 1.27, versus 0.82/0.30 for animating the whole panorama and 0.64/0.91 for animating perspective views separately.
- In a user agreement study, 70% of view-consistency judgments favored the proposed method versus 33% for separately animated perspective views.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use 4K4DGen instead of 3D-Cinemagraphy (circle) | Panorama-to-4D generation · Panorama test set | FID: 57.78 -> 16.59 | Helped |
| use 4K4DGen instead of 3D-Cinemagraphy (circle) | Panorama-to-4D generation · Panorama test set | KID (× 10⁻²): 1.3 -> 0.2 | Helped |
| use 4K4DGen instead of 3D-Cinemagraphy (circle) | Panorama-to-4D generation · Panorama test set | Q-Align (IQ): 0.48 -> 0.66 | Helped |
| use 4K4DGen instead of 3D-Cinemagraphy (circle) | Panorama-to-4D generation · Panorama test set | Q-Align (IA): 0.40 -> 0.44 | Helped |
| use 4K4DGen instead of 3D-Cinemagraphy (circle) | Panorama-to-4D generation · Panorama test set | Q-Align (VQ): 0.58 -> 0.62 | Helped |
| use 4K4DGen instead of 3D-Cinemagraphy (circle) | Panorama-to-4D generation · Panorama test set | Quality (UC): 12% -> 81% | Helped |
| animate the full panorama directly instead of using spherical latent fusion (ablation) | 4K4DGen animation · Panorama test set | Motion: 1.27 -> 0.30 | Hurt |
| animate perspective views separately without spherical latent fusion (ablation) | 4K4DGen animation · Panorama test set | View-consistency (UA): 70% -> 33% | Hurt |
| animate perspective views separately without spherical latent fusion (ablation) | 4K4DGen animation · Panorama test set | Motion: 1.27 -> 0.91 | Hurt |
| animate the full panorama directly instead of using spherical latent fusion (ablation) | 4K4DGen animation · Panorama test set | Q-Align (VQ): 0.82 -> 0.85 | No clear effect |
| animate perspective views separately without spherical latent fusion (ablation) | 4K4DGen animation · Panorama test set | Q-Align (VQ): 0.64 -> 0.85 | No clear effect |
| remove Spatial-Temporal Geometry Alignment (ablation) | 4K4DGen lifting · Panorama test set | geometric quality | Hurt |
Surprising findings
- Animating the full panorama with an I2V model yields higher Q-Align VQ (0.82) but very little motion (0.30), indicating quality scores alone do not capture animation.
Limitations
- Temporal animation quality depends on the pretrained image-to-video model.
- Cannot synthesize significant environmental changes such as weather or fireflies.
- High-resolution time-dependent Gaussian representation requires substantial storage.
Experimental setup
Datasets: SUN360 (used to fine-tune the text-to-panorama model that generated the test panoramas)
Metrics: FID KID Q-Align IQ Q-Align IA Q-Align VQ user choice user agreement motion
Baselines: 3D-Cinemagraphy (circle) 3D-Cinemagraphy (zoom-in) Animate Pano. Animate Pers.
Builds on: Animate-Anything Stable Video Diffusion 3D Gaussian Splatting MiDaS DreamScene360
Closest papers by meaning
- DynamicScaler: Seamless and Scalable Video Generation for Panoramic Scenes (CVPR 2025 · similarity 0.91)
- ViewPoint: Panoramic Video Generation with Pretrained Diffusion Models (NeurIPS 2025 · similarity 0.90)
- Scene4U: Hierarchical Layered 3D Scene Reconstruction from Single Panoramic Image for Your Immerse Exploration (CVPR 2025 · similarity 0.89)
- PanSplat: 4K Panorama Synthesis with Feed-Forward Gaussian Splatting (CVPR 2025 · similarity 0.88)
- PanoWan: Lifting Diffusion Video Generation Models to 360$^\circ$ with Latitude/Longitude-aware Mechanisms (NeurIPS 2025 · similarity 0.88)
- PanoWorld: Geometry-Consistent Panoramic Video World Modeling (NeurIPS 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.