Research map / 3D vision / Depth and stereo
Metric from Human: Zero-shot Monocular Metric Depth Estimation via Test-time Adaptation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Metric from Human (MfH), a test-time adaptation method that paints humans into an image with generative painting, recovers their metric size with an HMR model, and learns a per-image metric head on top of a relative depth model for zero-shot monocular metric depth estimation.
Problem
Monocular metric depth estimation (MMDE) models trained with supervision generalize poorly to unseen scenes because they depend on training scenes to predict metric scale, requiring large labeled datasets.
Approach
Use humans as scene-independent metric landmarks: generative painting inserts people into the input, HMR gives their metric sizes (SMPL shape space in meters), and these pseudo metric depths supervise a per-image affine metric head on top of a frozen relative depth model, so no metric annotations are needed.
Key results
- On NYUv2 zero-shot, MfH reaches δ1 83.2 and AbsRel 13.7, far above one-shot/zero-shot language-based baselines (Hu et al. 42.8 δ1, DepthCLIP 39.4 δ1).
- On KITTI, MfH achieves δ1 81.2 and AbsRel 13.3 without metric training, while many-shot UniDepth-V reaches 98.6 δ1.
- On ETH3D, MfH achieves AbsRel 24.0 versus 43.1 for UniDepth-V and 32.6 for ZeroDepth, showing better robustness across scene types; it is competitive but not best on NYUv2/KITTI.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply test-time adaptation with generated humans as metric landmarks | Depth Anything · NYUv2 | AbsRel: 38.8 -> 13.7 | Helped |
| Apply test-time adaptation with generated humans as metric landmarks | Depth Anything · KITTI | AbsRel: 47.3 -> 13.3 | Helped |
| Use generated-human test-time adaptation instead of the strongest many-shot baseline | MfH · DIODE (Indoor) | AbsRel: 18.1 -> 34.5 | Hurt |
| Use generated-human test-time adaptation instead of the strongest many-shot baseline | MfH · iBims-1 | AbsRel: 14.4 -> 23.3 | Hurt |
| Use generated-human test-time adaptation instead of the strongest many-shot baseline | MfH · ETH3D | AbsRel: 32.6 -> 24.0 | Helped |
| Replace true depth targets with painted depth targets (ablation) | ZoeDepth · NYUv2 | AbsRel: 22.4 -> 35.5 | Hurt |
| Replace true depth targets with painted depth targets (ablation) | Marigold · NYUv2 | AbsRel: 5.88 -> 21.4 | Hurt |
| Use painted disparity targets instead of true disparity targets (ablation) | Depth Anything · NYUv2 | AbsRel: 4.57 -> 21.9 | Hurt |
| Optimize all alignment and metric-head parameters sn, tn, s, t rather than only s, t (ablation) | MfH · NYUv2 | AbsRel: 28.0 -> 21.9 | Helped |
| Replace SIlog loss with l1 loss (ablation) | MfH · NYUv2 | AbsRel: 21.9 -> 201 | Hurt |
| Replace SIlog loss with l2 loss (ablation) | MfH · NYUv2 | AbsRel: 21.9 -> 27.1 | Hurt |
| Replace SIlog loss with MSElog loss (ablation) | MfH · NYUv2 | AbsRel: 21.9 -> 22.5 | Hurt |
Surprising findings
- Fully supervised MMDE models (e.g., UniDepth-V) can outperform the zero-shot MfH on KITTI and NYUv2 by large margins, while MfH wins on ETH3D.
- Using ground-truth disparity targets with Depth Anything gives AbsRel 4.57 on NYUv2, showing a large gap to the painted-human pseudo-labels (AbsRel 21.9).
Limitations
- Requires humans to be plausibly paintable in the scene, so it may fail on close-up scenes without room for people.
- Assumes MRDE predictions are affine-aligned with true depth; noisy MRDE outputs make the linear metric head inaccurate.
- Painted humans can be out-of-proportion or penetrating meshes, creating noisy pseudo ground truths; per-image optimization adds runtime proportional to number of painted images.
Experimental setup
Datasets: NYU-Depth V2 KITTI iBims-1 ETH3D DIODE
Benchmarks: NYU-Depth V2 KITTI DIODE (Indoor) iBims-1 ETH3D
Metrics: δ1 AbsRel SIlog RMSE
Baselines: ZoeDepth ZeroDepth Metric3D UniDepth-C UniDepth-V Depth Anything LORN DepthCLIP Hu et al.
Builds on: Depth Anything Stable Diffusion HMR 2.0 ZoeDepth SMPL
Closest papers by meaning
- UniDepth: Universal Monocular Metric Depth Estimation (CVPR 2024 · similarity 0.91)
- SharpDepth: Sharpening Metric Depth Predictions Using Diffusion Distillation (CVPR 2025 · similarity 0.88)
- Depth Pro: Sharp Monocular Metric Depth in Less Than a Second (ICLR 2025 · similarity 0.87)
- RSA: Resolving Scale Ambiguities in Monocular Depth Estimators through Language Descriptions (NeurIPS 2024 · similarity 0.87)
- BetterDepth: Plug-and-Play Diffusion Refiner for Zero-Shot Monocular Depth Estimation (NeurIPS 2024 · similarity 0.87)
- Repurposing Marigold for Zero-Shot Metric Depth Estimation via Defocus Blur Cues (NeurIPS 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.