Research map / 3D vision / Depth and stereo
Distilling Monocular Foundation Model for Fine-grained Depth Completion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DMD3C, a two-stage distillation framework that pre-trains depth completion networks on simulated LiDAR from monocular foundation model depth and then fine-tunes with an SSI loss, reaching first place on the KITTI depth completion leaderboard by RMSE.
Problem
Depth completion from sparse LiDAR is limited by sparse, incomplete ground-truth supervision, so models fail to learn fine-grained dense geometry; monocular depth foundation models offer dense supervision but suffer scale ambiguity.
Approach
Generate pre-training data from unlabeled natural images by predicting dense depth with a monocular foundation model, reconstructing a mesh from randomly sampled intrinsics, and simulating LiDAR via ray casting; then fine-tune on real data using L1 supervision plus a scale- and shift-invariant distillation loss with gradient matching regularization.
Key results
- Achieves 678.12 RMSE on the KITTI depth completion leaderboard, ranking first at submission time versus BP-Net's 684.90.
- On NYUv2, reports best RMSE (0.085) and best delta1.25 (99.7).
- Removing pre-training raises KITTI RMSE from 678.12 to 682.34; removing SSI loss raises it to 684.54.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add first-stage pre-training (ablation) | BP-Net depth completion · KITTI | RMSE: 682.34 -> 678.12 | Helped |
| add SSI Loss during fine-tuning (ablation) | BP-Net depth completion · KITTI | RMSE: 684.54 -> 678.12 | Helped |
| apply DMD3C training to LRRU (ablation) | LRRU · KITTI | RMSE: 696.51 -> 693.17 | Helped |
| apply DMD3C training to CFormer (ablation) | CFormer · KITTI | RMSE: 764.87 -> 760.29 | Helped |
| apply DMD3C training to BP-Net (ablation) | BP-Net · KITTI | RMSE: 684.90 -> 678.12 | Helped |
Limitations
- Monocular depth teacher suffers from scale and shift ambiguity, handled only through the SSI loss.
- Gains over prior methods on KITTI are small (e.g., 684.90 to 678.12 RMSE) and MAE/iMAE are not best.
Experimental setup
Datasets: COCO Google Landmarks nuScenes Cityscapes DAVIS KITTI Depth Completion NYU Depth V2
Benchmarks: KITTI Depth Completion NYU Depth V2 ScanNet DDAD VOID1500
Metrics: RMSE MAE iRMSE iMAE REL delta1.25
Baselines: S2D CSPN DeepLiDAR CSPN++ GuideNet FCFR ACMNet NLSPN PointDC RigNet DySPN BEV@DC
Builds on: Depth Anything V2 BP-Net MiDaS Poisson Surface Reconstruction
Closest papers by meaning
- Mind The Edge: Refining Depth Edges in Sparsely-Supervised Monocular Depth Estimation (CVPR 2024 · similarity 0.90)
- Flexible Depth Completion for Sparse and Varying Point Densities (CVPR 2024 · similarity 0.89)
- A Simple yet Universal Framework for Depth Completion (NeurIPS 2024 · similarity 0.89)
- Zero-shot Depth Completion via Test-time Alignment with Affine-invariant Depth Prior (AAAI 2025 · similarity 0.88)
- SharpDepth: Sharpening Metric Depth Predictions Using Diffusion Distillation (CVPR 2025 · similarity 0.88)
- Depth2Pose: A Pose-Based Benchmark for Monocular Depth Estimation without Ground-Truth Depth (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.