Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning
Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Scales monocular depth estimation by pseudo-labeling ~62M unlabeled images with a teacher trained on 1.5M labeled images, using strong perturbations (color distortion, CutMix) and DINOv2 feature alignment to build a zero-shot depth foundation model.
Problem
Monocular depth estimation generalizes poorly across domains because labeled depth datasets are limited in coverage; naive self-training with unlabeled images does not help when labeled data is already sufficient.
Approach
Use large-scale unlabeled images pseudo-labeled by a teacher, but train the student under strong perturbations (color jitter, blur, CutMix) to force learning of extra visual knowledge, and align student features to a frozen DINOv2 encoder with a tolerance-margin cosine loss to retain semantic priors.
Key results
- Zero-shot relative depth: ViT-L Depth Anything reaches AbsRel 0.076 on KITTI vs 0.127 for MiDaS v3.1 ViT-L, and 0.230 vs 0.251 on DDAD.
- Fine-tuned metric depth on NYUv2 reaches delta1 0.984 vs 0.964 for prior best VPD; on KITTI delta1 0.982 vs 0.978.
- Transferring the encoder to Cityscapes gives 86.2 mIoU (multi-scale) vs 84.6 for ConvNeXt-XL, and 59.4 mIoU on ADE20K vs 58.3 prior best.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add large-scale pseudo-labeled unlabeled images without strong perturbations (ablation) | Depth Anything · Sintel | AbsRel: 0.492 -> 0.481 | Helped |
| Add large-scale pseudo-labeled unlabeled images without strong perturbations (ablation) | Depth Anything · NYUv2 | AbsRel: 0.053 -> 0.054 | Hurt |
| Apply strong perturbations to unlabeled images during student training (ablation) | Depth Anything with unlabeled images · KITTI, NYUv2, Sintel, DDAD, ETH3D, DIODE | AbsRel: KITTI 0.085 -> 0.081; NYUv2 0.054 -> 0.048; Sintel 0.481 -> 0.469; DDAD 0.242 -> 0.235; ETH3D 0.138 -> 0.134; DIODE 0.07 | Helped |
| Add DINOv2 feature-alignment semantic constraint (ablation) | Depth Anything with perturbed unlabeled training · KITTI, NYUv2, Sintel, DDAD, ETH3D, DIODE | AbsRel: KITTI 0.081 -> 0.076; NYUv2 0.048 -> 0.043; Sintel 0.469 -> 0.458; DDAD 0.235 -> 0.230; ETH3D 0.134 -> 0.127; DIODE 0.06 | Helped |
| Use Depth Anything ViT-L instead of MiDaS v3.1 ViT-L | Zero-shot relative depth estimation · DDAD | AbsRel: 0.251 -> 0.230 | Helped |
| Initialize metric-depth fine-tuning with Depth Anything instead of VPD baseline | Metric depth estimation · NYUv2 | AbsRel: 0.069 -> 0.056 | Helped |
| Replace ZoeDepth's MiDaS encoder with Depth Anything encoder | ZoeDepth metric-depth model · Virtual KITTI 2 | AbsRel: 0.106 -> 0.085 | Helped |
Surprising findings
- Naively adding pseudo-labeled unlabeled images did not improve over labeled-only training when labeled data was already sufficient.
- Adding a semantic segmentation auxiliary task (RAM+GroundingDINO+HQ-SAM labels) failed to boost the depth model, whereas feature alignment to DINOv2 helped.
Limitations
- Auxiliary semantic segmentation supervision did not yield gains in the authors' trials, so the semantic-alignment choice is heuristic.
- Metric depth experiments use a relatively basic ZoeDepth-based depth model, which the authors note may leave gains unexplored.
- Relies on a teacher's pseudo-labels, so errors in the teacher propagate to the student.
Experimental setup
Datasets: BlendedMVS DIML HRWSI IRS MegaDepth TartanAir BDD100K Google Landmarks ImageNet-21K LSUN Objects365 Open Images V7
Benchmarks: KITTI NYUv2 Sintel DDAD ETH3D DIODE Cityscapes ADE20K SUN RGB-D iBims-1 HyperSim Virtual KITTI 2
Metrics: AbsRel delta1 RMSE mIoU
Baselines: MiDaS v3.1 ZoeDepth VPD AiT SwinV2-L DINOv2
Builds on: MiDaS DINOv2 DPT ZoeDepth CutMix
Closest papers by meaning
- Depth Anything V2 (NeurIPS 2024 · similarity 0.93)
- Video Depth Anything: Consistent Depth Estimation for Super-Long Videos (CVPR 2025 · similarity 0.90)
- A Simple yet Universal Framework for Depth Completion (NeurIPS 2024 · similarity 0.90)
- Depth Anywhere: Enhancing 360 Monocular Depth Estimation via Perspective Distillation and Unlabeled Data Augmentation (NeurIPS 2024 · similarity 0.89)
- Depth Pro: Sharp Monocular Metric Depth in Less Than a Second (ICLR 2025 · similarity 0.89)
- DME: Unveiling the Bias for Better Generalized Monocular Depth Estimation (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.