Research map / 3D vision / Depth and stereo
Scalable Autoregressive Monocular Depth Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DAR, a depth autoregressive Transformer that combines low-to-high resolution token-map prediction (patch-wise causal mask) with coarse-to-fine bin refinement (Multiway Tree Bins) for monocular depth estimation, scaling to 2.0B parameters.
Problem
Monocular depth estimation has largely used encoder-decoder regression or ordinal-regression methods; the paper asks whether an autoregressive model can be designed for MDE and scaled effectively.
Approach
Recasts MDE as two autoregressive objectives: a resolution objective (next-resolution token map predicted with a patch-wise causal mask, in the style of next-scale prediction) and a granularity objective (recursively refined bins, Multiway Tree Bins, injected back into token maps via Bins Injection).
Key results
- DAR-Large (2.0B) reaches RMSE 1.799 on KITTI vs 1.896 for Depth Anything, and 0.205 RMSE / 0.982 δ1 on NYU Depth V2.
- DAR-Small (440M) outperforms Depth Anything on KITTI (RMSE 1.839 vs 1.896) at similar size.
- Zero-shot on SUN RGB-D after training only on NYU Depth V2: Abs Rel 0.112 and δ1 0.885, better than Depth Anything (0.119, 0.864).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DAR-Large instead of EcoDepth | NYU Depth v2 depth estimation · NYU Depth v2 test | RMSE: 0.218 -> 0.205 | Helped |
| Use DAR-Large instead of EcoDepth | NYU Depth v2 depth estimation · NYU Depth v2 test | δ1: 0.978 -> 0.982 | Helped |
| Use DAR-Large instead of Depth Anything | KITTI depth estimation · KITTI | RMSE: 1.896 -> 1.799 | Helped |
| Use DAR-Large instead of Depth Anything | KITTI depth estimation · KITTI | Abs Rel: 0.046 -> 0.044 | Helped |
| Use DAR instead of Depth Anything without SUN RGB-D fine-tuning | zero-shot depth estimation trained on NYU Depth v2 · SUN RGB-D | RMSE: 0.346 -> 0.319 | Helped |
| Add MTBins and Bins Injection (ablation) | baseline with Transformer · NYU Depth v2 | RMSE: 0.229 -> 0.220 | Helped |
| Add the full DAR framework (ablation) | baseline with MTBins and Bins Injection · NYU Depth v2 | RMSE: 0.220 -> 0.217 | Helped |
| Scale DAR up to 2.0B parameters (ablation) | Baseline + DAR · NYU Depth v2 | RMSE: 0.217 -> 0.205 | Helped |
Surprising findings
- Trained only on NYU Depth V2, DAR generalizes zero-shot to SUN RGB-D better than Depth Anything, which was pretrained on 62M images.
Limitations
- Multi-step progressive prediction may blur object boundaries and reduce sharpness.
- Autoregressive Transformer makes the model parameter count high, especially when scaled up.
Experimental setup
Datasets: NYU Depth V2 KITTI
Benchmarks: NYU Depth V2 KITTI SUN RGB-D
Metrics: Abs Rel RMSE Sq Rel log10 RMSE log δ1 δ2 δ3
Baselines: Depth Anything EcoDepth ZoeDepth VPD IEBins BinsFormer PixelFormer NeWCRFs
Builds on: VAR Depth Anything Ord2Seq DORN ConvGRU
Closest papers by meaning
- DME: Unveiling the Bias for Better Generalized Monocular Depth Estimation (AAAI 2024 · similarity 0.88)
- Depth2Pose: A Pose-Based Benchmark for Monocular Depth Estimation without Ground-Truth Depth (NeurIPS 2026 · similarity 0.88)
- Depth Pro: Sharp Monocular Metric Depth in Less Than a Second (ICLR 2025 · similarity 0.87)
- Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation (CVPR 2024 · similarity 0.87)
- DepthMaster: Unified Monocular Depth Estimation for Perspective and Panoramic Images (NeurIPS 2026 · similarity 0.87)
- Depth Anything V2 (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.