Research map / Image generation / Fast diffusion sampling
DepthFM: Fast Generative Monocular Depth Estimation with Flow Matching
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Frames monocular depth estimation as direct flow-matching transport from image latents to depth latents, initialized from Stable Diffusion 2.1 and augmented with synthetic and Metric3D-labeled data, enabling fast few-step generative depth prediction.
Problem
Discriminative depth estimators produce blurry edges, while diffusion-based generative depth estimators are slow due to curved noise-to-depth ODE trajectories and need large training data.
Approach
Replace noise-to-depth diffusion with a flow-matching transport starting directly from the image latent (paired image-depth coupling), fine-tune from a diffusion image prior, add noise augmentation on the image latent, and mix in discriminative teacher-labeled pseudo-depth data.
Key results
- DepthFM-ID reaches 96.3% delta1 on NYUv2 zero-shot, comparable to Marigold/GeoWizard's ~96-97%, and DIODE AbsRel 0.212 vs Marigold 0.308.
- With 1 function evaluation DepthFM achieves 95.0% delta1 on NYUv2, above Marigold's 82.7% at 4 steps.
- On Middlebury-2014 edge evaluation DepthFM gets 33.54% edge precision and 49.31% recall, above Depth Anything v2 (31.67%, 40.25%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use image-to-depth transport rather than noise-to-depth transport (ablation) | DepthFM · NYUv2 | δ1: 92.4 -> 94.6 | Helped |
| fine-tune the full image prior rather than train from scratch (ablation) | DepthFM · NYUv2 | δ1: 80.0 -> 95.5 | Helped |
| fine-tune the full image prior rather than use LoRA rank 8 (ablation) | DepthFM · NYUv2 | δ1: 75.6 -> 95.5 | Helped |
| fine-tune the full image prior rather than train from scratch (ablation) | DepthFM · DIODE | δ1: 68.8 -> 78.5 | Helped |
| fine-tune the full image prior rather than use LoRA rank 8 (ablation) | DepthFM · DIODE | δ1: 70.0 -> 78.5 | Helped |
| add the image prior (ablation) | DepthFM · NYUv2 | δ1: 80.0 -> 95.5 | Helped |
| add the depth prior to the image-prior model (ablation) | DepthFM · NYUv2 | δ1: 95.5 -> 96.3 | Helped |
| increase noise augmentation level from ts = 0.1 to ts = 0.4 (ablation) | DepthFM · NYUv2 | δ1: 93.7 -> 95.5 | Helped |
| increase noise augmentation level from ts = 0.4 to ts = 0.8 (ablation) | DepthFM · NYUv2 | δ1: 95.5 -> 95.4 | No clear effect |
| use image-depth paired coupling rather than random coupling (ablation) | DepthFM · coupling distance | EMD-L2: 0.981 -> 0.686 | Helped |
| use image-depth paired coupling rather than random coupling (ablation) | DepthFM · coupling distance | EMD-L1: 0.974 -> 0.691 | Helped |
| add 7.4K discriminative image-depth samples (ablation) | DepthFM-I · NYUv2 | δ1: 95.5 -> 96.3 | Helped |
Surprising findings
- Starting flow from the image latent rather than Gaussian noise gives much higher accuracy at low NFE, and a LoRA adapter (rank 8) performs far worse than full fine-tuning (75.6 vs 95.5 on NYUv2).
Limitations
- Ensembling for confidence multiplies inference cost (default ensemble size 10).
- Discriminative teacher pseudo-labels trade accuracy against blurrier fidelity when mixed too heavily.
- Zero-shot results still trail some discriminative models such as Depth Anything on KITTI and ETH3D.
Experimental setup
Datasets: Hypersim Virtual KITTI Unsplash NYUv2 KITTI ETH3D ScanNet DIODE Middlebury-2014
Benchmarks: NYUv2 KITTI ETH3D ScanNet DIODE Middlebury-2014 NYUv2 depth completion
Metrics: AbsRel delta1 accuracy RMSE edge precision edge recall
Baselines: Marigold GeoWizard Depth Anything Depth Anything v2 Metric3D v2 MiDaS Omnidata HDN DPT NLSPN CompletionFormer
Builds on: flow matching Marigold Stable Diffusion 2.1 Metric3D v2 optimal transport conditional flow matching
Closest papers by meaning
- Repurposing Diffusion-Based Image Generators for Monocular Depth Estimation (CVPR 2024 · similarity 0.90)
- SharpDepth: Sharpening Metric Depth Predictions Using Diffusion Distillation (CVPR 2025 · similarity 0.90)
- Depth Any Video with Scalable Synthetic Data (ICLR 2025 · similarity 0.89)
- Zero-shot Depth Completion via Test-time Alignment with Affine-invariant Depth Prior (AAAI 2025 · similarity 0.88)
- More Than Generation: Unifying Generation and Depth Estimation via Text-to-Image Diffusion Models (NeurIPS 2025 · similarity 0.88)
- Depth Pro: Sharp Monocular Metric Depth in Less Than a Second (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.