Research map / Video / Video generation

PanoWorld: Geometry-Consistent Panoramic Video World Modeling

NeurIPS 2026 · Video › Video generation · approach: World-model video generation · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Adapts a pretrained perspective video world model (Cosmos Predict 2.5) into a panoramic video world model that generates geometry-consistent 360° video from a perspective image and caption, using depth and 3D trajectory consistency losses plus spherical adaptations, and releases the PanoGeo dataset.

Problem

Panoramic (equirectangular) video generators optimize visual realism but produce outputs with inconsistent depth, broken point correspondences, and implausible motion across the sphere, which blocks full-sphere world modeling for embodied AI.

Approach

Fine-tune a perspective video world model for ERP panoramas with two lightweight geometry regularizers on the predicted clean latent: a depth L1+gradient loss against DAP pseudo-depth and a trajectory loss that lifts CoTracker3 tracks to 3D via predicted depth and supervises position, velocity and acceleration; plus latitude-aware RoPE, cos-latitude loss weighting, circular shift augmentation, and

Key results

• PanoWorld reaches FVD 56.1 (-13.7% vs ARGUS) and FID 136.4 (vs Imagine360 142.9) on the 150-clip Stage 1 suite, while lowest 3D-Smooth (0.025 m, -22% vs ARGUS) and Depth-sigma (0.013, -38%).
• Against OmniRoam, PanoWorld's 3D-Smooth is far lower (0.025 vs 4.08) and Tr-Life is 0.994 vs roughly 0.23.
• Ablation: adding both losses halves 3D-Smooth (0.051 to 0.025) and Depth-sigma (0.028 to 0.013) relative to no-loss baseline with FVD roughly unchanged.

Surprising findings

• ARGUS matches PanoWorld's smoothness scores mainly by producing quasi-static rollouts, so smoothness alone can be gamed.
• Follow-Your-Canvas attains higher CLIP-T than PanoWorld (0.279 vs 0.264) because it keeps the camera on the prompt-relevant input crop.

Limitations

• Fixed 93-frame clips; long rollouts accumulate drift and are left to future work.
• Monocular depth pseudo-labels (DAP) and the perspective Cosmos prior introduce supervisory bias on ERP content.
• Geometric metrics are computed using re-run DAP and CoTracker3, so they depend on the same pseudo-label models used for training supervision.

Experimental setup

Datasets: PanoGeo (self-captured 4K 360 video) x360 360DVD Argus YouTube test split Habitat-Sim renders

Benchmarks: PanoGeo held-out Argus testset Habitat-Sim

Metrics: FVD FAED FID CLIP-T 3D-Smooth Depth-sigma Tr-Life

Baselines: 360DVD Imagine360 ARGUS Follow-Your-Canvas OmniRoam

Builds on: Cosmos Predict 2.5 rectified flow Depth Any Panoramas CoTracker3 SigLIP2

Closest papers by meaning

• ViewPoint: Panoramic Video Generation with Pretrained Diffusion Models (NeurIPS 2025 · similarity 0.92)
• PanoWan: Lifting Diffusion Video Generation Models to 360$^\circ$ with Latitude/Longitude-aware Mechanisms (NeurIPS 2025 · similarity 0.90)
• Splatter-360: Generalizable 360 Gaussian Splatting for Wide-baseline Panoramic Images (CVPR 2025 · similarity 0.88)
• DynamicScaler: Seamless and Scalable Video Generation for Panoramic Scenes (CVPR 2025 · similarity 0.88)
• 4K4DGen: Panoramic 4D Generation at 4K Resolution (ICLR 2025 · similarity 0.88)
• SimVS: Simulating World Inconsistencies for Robust View Synthesis (CVPR 2025 · similarity 0.87)