Research map / 3D vision / Depth and stereo
UniDepth: Universal Monocular Metric Depth Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes UniDepth, a universal monocular metric depth estimator that predicts metric 3D points from a single image without camera intrinsics, using a self-prompted dense camera module, a pseudo-spherical output space, and a geometric invariance loss.
Problem
Monocular metric depth estimation methods overfit to their training domains and camera intrinsics, failing to generalize zero-shot to unseen domains and arbitrary cameras without external camera information.
Approach
Predict a dense camera representation (azimuth and elevation per pixel) from the image itself to prompt the depth module via cross-attention, output 3D points in a pseudo-spherical space (θ, ϕ, log-depth) that disentangles camera and depth, and enforce consistency of camera-conditioned depth features across geometric augmentations.
Key results
- In zero-shot evaluation on ten datasets, UniDepth-V reaches 98.4 δ1 on NYU and 93.4 δ0.5 / 98.6 δ1 on KITTI Eigen-split, ranking first on the official KITTI Depth Prediction Benchmark at submission time.
- UniDepth achieves an average 34.0% improvement in SIlog over prior SotA methods across zero-shot test sets, with UniDepth-V outperforming in-domain trained methods on NYU and KITTI.
- Ablations show that removing the pseudo-spherical representation (Cartesian output) drops out-of-domain δ1 from 57.06 to 37.09, and removing stop-gradient on the camera module drops out-of-domain δ1 to 46.60.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use UniDepth-V instead of ZoeDepth | UniDepth-V · NYU test set | δ1: 95.2 -> 98.4 | Helped |
| use UniDepth-V instead of NeWCRF | UniDepth-V · KITTI Eigen-split test set | δ1: 97.5 -> 98.6 | Helped |
| add the camera module (ablation) | Full UniDepth · Out-of-Domain | δ1: 48.38±0.97 -> 57.06±1.48 | Helped |
| use pseudo-spherical rather than Cartesian output (ablation) | Full UniDepth · Out-of-Domain | δ1: 37.09±1.37 -> 57.06±1.48 | Helped |
| add geometric invariance loss (ablation) | Full UniDepth · Out-of-Domain | δ1: 52.89±0.21 -> 57.06±1.48 | Helped |
| use dense instead of sparse camera prompting (ablation) | Full UniDepth · Out-of-Domain | δ1: 55.65±0.18 -> 57.06±1.48 | Helped |
| stop gradients from the camera module to the encoder (ablation) | Full UniDepth · Out-of-Domain | δ1: 46.60±0.25 -> 57.06±1.48 | Helped |
| add camera module and use Cartesian 3D output (ablation) | Full UniDepth · Out-of-Domain | δ1: 48.19±1.02 -> 57.06±1.48 | Helped |
| add camera module to the pseudo-spherical baseline (ablation) | Full UniDepth · Out-of-Domain | δ1: 51.22±0.35 -> 57.06±1.48 | Helped |
Surprising findings
- Predicting camera with a bootstrapped self-prompt can outperform models given ground-truth camera intrinsics on some datasets such as ETH3D and IBims-1 in SIlog.
- Zero-shot UniDepth surpasses methods trained directly on NYU and KITTI in δ1 on those benchmarks.
Limitations
- Scale-dependent metrics degrade on some out-of-domain sets such as ETH3D and IBims-1 (FA drops of 11.8% and 31.4%), suggesting need for domain-specific fine-tuning.
- Bootstrapped camera prediction adds noise to scaled depth on out-of-domain camera types such as ETH3D and HAMMER, worsening δ1.
- Training requires roughly 12 days on 8 NVIDIA A100 GPUs.
Experimental setup
Datasets: Argoverse2 Waymo DrivingStereo Cityscapes BDD100K Mapillary-PSD A2D2 ScanNet Taskonomy SUN-RGBD Diode Indoor IBims-1
Benchmarks: SUN-RGBD Diode Indoor IBims-1 VOID HAMMER ETH-3D nuScenes DDAD NYU-Depth V2 KITTI Eigen-split
Metrics: δ1 SIlog F-score A.Rel RMS RMSlog Chamfer Distance δ0.5 ARelC
Baselines: BTS AdaBins NeWCRF iDisc ZoeDepth Metric3D ZeroDepth
Builds on: ViT ConvNext DINOv2
Closest papers by meaning
- UniK3D: Universal Camera Monocular 3D Estimation (CVPR 2025 · similarity 0.91)
- Metric from Human: Zero-shot Monocular Metric Depth Estimation via Test-time Adaptation (NeurIPS 2024 · similarity 0.91)
- SharpDepth: Sharpening Metric Depth Predictions Using Diffusion Distillation (CVPR 2025 · similarity 0.90)
- A Simple yet Universal Framework for Depth Completion (NeurIPS 2024 · similarity 0.89)
- DepthMaster: Unified Monocular Depth Estimation for Perspective and Panoramic Images (NeurIPS 2026 · similarity 0.89)
- GeoDepth: From Point-to-Depth to Plane-to-Depth Modeling for Self-Supervised Monocular Depth Estimation (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.