Research map / Label-efficient and robust learning / Recognition and long-tail
Depth Anything V2
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Depth Anything V2 trains a DINOv2-G teacher purely on precise synthetic depth data, pseudo-labels 62M unlabeled real images with it, and distills students (25M to 1.3B) on those pseudo-labels, plus introduces the DA-2K relative-depth benchmark.
Problem
Monocular depth estimation models trained on noisy real labeled data produce coarse depth and fail on fine details and transparent/reflective surfaces, while synthetic-only training suffers from distribution shift and limited scene coverage; existing test benchmarks are also noisy, low-diversity, an
Approach
Replace all labeled real images with precise synthetic depth labels for a large teacher, bridge the synthetic-to-real gap by pseudo-labeling large-scale unlabeled real images with the teacher, and distill to smaller students via label-level (prediction-level) distillation rather than feature-level distillation.
Key results
- On the DA-2K benchmark, the ViT-G model reaches 97.4% accuracy versus 86.8% for Marigold and 88.5% for Depth Anything V1; even ViT-S reaches 95.3%.
- Students trained only on 62M pseudo-labeled real images improve over synthetic-only ViT-L training (DA-2K 96.0% to 97.1%, KITTI AbsRel 0.081 to 0.074).
- Metric fine-tuning of the ViT-L encoder achieves KITTI AbsRel 0.045 and NYU-D delta1 0.983 (ZoeDepth-style pipeline), and ViT-S already beats ViT-L baselines.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add pseudo-labeled real images to synthetic-only student training (ablation) | Depth Anything V2 ViT-S · DA-2K | accuracy (%): 89.8 -> 95.3 | Helped |
| add pseudo-labeled real images to synthetic-only student training (ablation) | Depth Anything V2 ViT-B · DA-2K | accuracy (%): 92.9 -> 97.0 | Helped |
| add pseudo-labeled real images to synthetic-only student training (ablation) | Depth Anything V2 ViT-L · DA-2K | accuracy (%): 96.0 -> 97.1 | Helped |
| remove synthetic labeled images from student training (ablation) | Depth Anything V2 ViT-S · DA-2K | accuracy (%): 94.1 -> 95.3 | Helped |
| remove synthetic labeled images from student training (ablation) | Depth Anything V2 ViT-B · DA-2K | accuracy (%): 96.7 -> 97.0 | Helped |
| replace DIML manual labels with pseudo labels (ablation) | Depth Anything V2 transfer model trained on DIML · DA-2K | accuracy (%): 80.2 -> 89.7 | Helped |
| use all eight diverse unlabeled datasets instead of SA-1B alone (ablation) | Depth Anything V2 ViT-S · KITTI | AbsRel: 0.090 -> 0.085 | Helped |
| increase the gradient matching loss weight on synthetic training data (ablation) | Depth Anything V2 teacher · Synthetic-data depth sharpness | sharpness: 0.5 to 4.0 | Helped |
| apply gradient matching loss to training on labeled real datasets (ablation) | Depth Anything V2 · Labeled real datasets | depth sharpness | No clear effect |
| add real labeled images (e.g., HRWSI) to synthetic training data (ablation) | Depth Anything V2 teacher · Fine-grained depth predictions | depth sharpness | Hurt |
| scale test-time resolution from the base resolution to 2× and 4× (ablation) | Depth Anything V2 · Test-time depth prediction | depth sharpness: 2× and 4× of the base resolution (518) | Helped |
| replace DINOv2-G with DINOv2-G Reg (ablation) | Synthetic-only teacher model · Sintel | AbsRel: 0.530 -> 0.753 | Hurt |
Surprising findings
- Solely from standard zero-shot metrics on five benchmarks, Depth Anything V2 is comparable to or slightly worse than V1, because the benchmarks are noisy and do not reflect fine-detail quality.
- Adding the real dataset HRWSI, even at 5% of training images, to synthetic data ruins fine-grained predictions.
- DINOv2-G with registers is much inferior to the non-register DINOv2-G when trained on synthetic data.
Limitations
- Training on 62M unlabeled images is computationally very heavy.
- Current synthetic training sets are not diverse enough, limiting the teacher.
- DA-2K provides sparse relative depth pairs and does not replace dense benchmarks.
Experimental setup
Datasets: BlendedMVS Hypersim IRS TartanAir Virtual KITTI 2 BDD100K Google Landmarks ImageNet-21K LSUN Objects365 Open Images V7 Places365
Benchmarks: KITTI NYU-D Sintel ETH3D DIODE DA-2K Cityscapes ADE20K NTIRE 2024 Transparent Surface Challenge
Metrics: AbsRel delta1 delta2 delta3 RMSE log10 accuracy (%) mIoU
Baselines: MiDaS V3.1 Depth Anything V1 Marigold GeoWizard DepthFM ZoeDepth Metric3D
Builds on: Depth Anything V1 DINOv2 MiDaS DPT ZoeDepth
Closest papers by meaning
- Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data (CVPR 2024 · similarity 0.93)
- Video Depth Anything: Consistent Depth Estimation for Super-Long Videos (CVPR 2025 · similarity 0.90)
- Depth Pro: Sharp Monocular Metric Depth in Less Than a Second (ICLR 2025 · similarity 0.89)
- Depth Anywhere: Enhancing 360 Monocular Depth Estimation via Perspective Distillation and Unlabeled Data Augmentation (NeurIPS 2024 · similarity 0.88)
- DepthCues: Evaluating Monocular Depth Perception in Large Vision Models (CVPR 2025 · similarity 0.88)
- Depth Any Video with Scalable Synthetic Data (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.