Research map / 3D vision / Depth and stereo
Depth Prompting for Sensor-Agnostic Depth Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a depth prompt module that plugs into a frozen monocular depth foundation model (DepthFormer backbone, bias-tuned) to turn sparse depth from any sensor into metric dense depth via learned adaptive affinity for spatial propagation.
Problem
Sparse-depth-guided dense depth estimation fails to generalize across sensor biases: sparsity/density, scan pattern, and scan range, so models trained on one sensor do not transfer to others.
Approach
Reinterpret prompt learning as spatial propagation: a depth prompt encoder-decoder learns an adaptive affinity map from sparse depth and image features, while a frozen monocular foundation model (with bias tuning) supplies relative depth that a least-squares solver scales to absolute metric depth using the sparse measurements.
Key results
- On NYUv2 with 1 depth point, the method reaches RMSE 0.4040 m / MAE 0.2848 m / DELTA1 0.8935, while the best baseline (NLSPN) is at 1.1929 / 0.9521 / 0.3616 in Table 1 (the reported result is from the paper's table).
- On KITTI DC with 1 scan line, RMSE 2.3841 m vs 6.0305 m for NLSPN (Table 2 as reported).
- Under range bias (KITTI 15m~80m to 0m~15m), RMSE 7.648 vs 11.283 for CSPN and 11.492 for CompletionFormer (Table 5).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use depth prompting with a pretrained DepthFormer foundation model | Sparse depth estimation · NYUv2, 1 sample | RMSE: 1.1093 -> 0.4040 | Helped |
| Use depth prompting with a pretrained DepthFormer foundation model | Sparse depth estimation · KITTI DC, 1 line | RMSE: 4.4444 -> 2.8234 | Helped |
| Add random depth augmentation (ablation) | Depth prompting · NYUv2 sparsity bias | RMSE: 0.498 -> 0.231 | Helped |
| Add spatial propagation (ablation) | Depth prompting · NYUv2 pattern bias | RMSE: 0.145 -> 0.108 | Helped |
| Add the depth prompt module (ablation) | Depth prompting · NYUv2 pattern bias | RMSE: 0.301 -> 0.108 | Helped |
| Use a pretrained depth foundation model (ablation) | Depth prompting · NYUv2 range bias | RMSE: 1.283 -> 0.206 | Helped |
| Use the least-squares solver (ablation) | Depth prompting · NYUv2 range bias | RMSE: 0.520 -> 0.206 | Helped |
| Replace DepthFormer with MiDaS (ablation) | Depth prompting · NYUv2 sparsity bias | RMSE: 0.3997 -> 0.4472 | Hurt |
| Replace DepthFormer with KBR (ablation) | Depth prompting · NYUv2 sparsity bias | RMSE: 0.3997 -> 0.3632 | Helped |
| Apply random depth augmentation to NLSPN and CompletionFormer (ablation) | NLSPN and CompletionFormer · NYUv2 and KITTI DC sparse-input settings | RMSE | Helped |
Surprising findings
- The self-supervised KBR backbone outperforms the supervised MiDaS backbone within the same framework, which the authors attribute to a more generalizable feature space.
Limitations
- Spatial propagation cannot produce depth where no initial seeds exist, so a least-squares global scale fit is required as a workaround.
- Few-shot adaptation to new indoor/outdoor domains still requires ground-truth annotations of 10 to 100 images.
Experimental setup
Datasets: NYUv2 KITTI DC VOID NuScenes
Benchmarks: NYUv2 KITTI DC VOID NuScenes
Metrics: RMSE MAE DELTA1
Baselines: CSPN S2D NLSPN DySPN CostDCNet CompletionFormer SAN Ours+MiDaS Ours+KBR
Builds on: DepthFormer CompletionFormer NLSPN SAM prompt encoder bias tuning (BitFit)
Closest papers by meaning
- Robust Depth Enhancement via Polarization Prompt Fusion Tuning (CVPR 2024 · similarity 0.89)
- Prompting Depth Anything for 4K Resolution Accurate Metric Depth Estimation (CVPR 2025 · similarity 0.89)
- A Simple yet Universal Framework for Depth Completion (NeurIPS 2024 · similarity 0.88)
- DepthCues: Evaluating Monocular Depth Perception in Large Vision Models (CVPR 2025 · similarity 0.87)
- Depth Pro: Sharp Monocular Metric Depth in Less Than a Second (ICLR 2025 · similarity 0.87)
- Zero-shot Depth Completion via Test-time Alignment with Affine-invariant Depth Prior (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.