Research map / 3D vision / Depth and stereo

Unified Panoramic Geometry Estimation via Multi-View Foundation Models

NeurIPS 2026 · 3D vision › Depth and stereo · approach: Cost-volume multi-view stereo · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Adapts the Depth Anything 3 multi-view transformer to panoramas via cubemap faces with camera conditioning, cross-face padding, and mixed panoramic/perspective training, yielding a unified model for SI depth, metric depth, normals, and sky masks.

Problem

Panoramic geometry estimation lacks large, metrically accurate training data and suffers from equirectangular distortion, so models overfit and fail to generalize; perspective foundation geometry models cannot be directly applied to 360° images.

Approach

Represent panoramas as six cubemap faces fed to a pre-trained multi-view feed-forward geometry transformer (DA3) with fixed camera conditioning, add cross-face padding in the decoder, and co-train on synthetic panoramas and real perspective images, with coupled multi-task heads for SI depth, metric scale, normals and sky.

Key results

• On Matterport3D360 scale-invariant depth, PaGeR reaches AbsRel 9.67 vs 11.06 for the next best DA^2.
• On ZüriPano outdoor SI depth, AbsRel drops from 18.27 (RPG360) to 9.36.
• Surface normals on Structured3D: mean angular error 5.49° and MSE 174.9 versus prior best MSE 246.6.

Surprising findings

• Specialized single-task models outperform the unified model on most metrics, with the largest gap for metric depth (e.g., Matterport3D360 AbsRel 17.71 vs 21.83).
• Allowing metric head gradients into the shared ViT degraded SI depth and normal quality, so the backbone is frozen for the metric head.

Limitations

• Unreliable on specular, reflective, or transparent surfaces and material ambiguities.
• Subtle misalignments can persist at cubemap face boundaries in structurally complex scenes.
• Metric depth lags behind specialized metric models because the backbone is frozen for the metric head.

Experimental setup

Datasets: Structured3D PanoInfinigen ScanNet++ ARKitScenes Matterport3D360 Stanford2D3DS ZüriPano

Benchmarks: Matterport3D360 Stanford2D3DS ZüriPano Structured3D Replica360_4K

Metrics: AbsRel RMSE delta1 Mean angular error MSE delta_5 delta_22.5 Seam Defect Density Seam Prevalence Seam Severity

Baselines: DreamCube DepthAnyCamera MoGe EGformer DAP RPG360 UniK3D PanDA DA^2 PanoNormal HyperSphere MTL

Builds on: Depth Anything 3 DINOv2 DPT Infinigen AdamW

Closest papers by meaning

• DepthMaster: Unified Monocular Depth Estimation for Perspective and Panoramic Images (NeurIPS 2026 · similarity 0.90)
• Splatter-360: Generalizable 360 Gaussian Splatting for Wide-baseline Panoramic Images (CVPR 2025 · similarity 0.89)
• PanDA: Towards Panoramic Depth Anything with Unlabeled Panoramas and Mobius Spatial Augmentation (CVPR 2025 · similarity 0.87)
• PanoWorld: Geometry-Consistent Panoramic Video World Modeling (NeurIPS 2026 · similarity 0.86)
• Fully Geometric Panoramic Localization (CVPR 2024 · similarity 0.86)
• Scene4U: Hierarchical Layered 3D Scene Reconstruction from Single Panoramic Image for Your Immerse Exploration (CVPR 2025 · similarity 0.86)