Research map / 3D vision / Depth and stereo
From-Ground-To-Objects: Coarse-to-Fine Self-supervised Monocular Depth Estimation of Dynamic Objects with Ground Contact Prior
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a coarse-to-fine self-supervised monocular depth training strategy that uses a ground-contact prior (GDS-Loss) and masks dynamic-class objects, then refines with a cost-volume-weighted regularization, improving several existing depth networks on Cityscapes and KITTI.
Problem
Self-supervised monocular depth estimation learns inaccurate depth for moving objects that violate the static-scene assumption, since the reprojection loss can be minimized with wrong object depth and motion.
Approach
Exclude dynamic-class objects from the reprojection loss in a coarse stage and supervise their depth via a Ground-contacting-prior Disparity Smoothness Loss that ties object disparity to its ground contact points; then fine-tune with the full reprojection loss plus a regularization loss weighted by a cost-volume-based depth discrepancy factor. The strategy plugs into existing DE networks without a
Key results
- On Cityscapes at 192x640, Ours-MonoViT reaches abs rel 0.088 vs 0.106 for MonoViT (KITTI 0.096 vs 0.099 for MonoViT).
- Ours-Monodepth2 improves Cityscapes abs rel from 0.125 to 0.102 at 192x640, and in the dynamic-object regions from 0.185 to 0.119.
- At 128x416 on Cityscapes, Ours-MonoViT (abs rel 0.096) outperforms multi-frame DynamicDepth (0.103).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply coarse-to-fine ground-contact-prior training to MonoViT | MonoViT · Cityscapes, 192 × 640 | abs rel: 0.106 -> 0.088 | Helped |
| Apply coarse-to-fine ground-contact-prior training to MonoViT | MonoViT · KITTI Eigen split, 192 × 640 | abs rel: 0.099 -> 0.096 | Helped |
| Apply coarse-to-fine ground-contact-prior training to MonoViT | MonoViT · Cityscapes dynamic objects, 128 × 416 | abs rel: 0.142 -> 0.109 | Helped |
| Exclude dynamic-class objects from reprojection loss (ablation) | Monodepth2 coarse training · Cityscapes WIR | abs rel: 0.125 -> 0.119 | Helped |
| Exclude dynamic-class objects from reprojection loss (ablation) | Monodepth2 coarse training · Cityscapes DOR | abs rel: 0.185 -> 0.160 | Helped |
| Add GDS-Loss to masked reprojection training (ablation) | Monodepth2 coarse training · Cityscapes WIR | abs rel: 0.119 -> 0.104 | Helped |
| Add GDS-Loss to masked reprojection training (ablation) | Monodepth2 coarse training · Cityscapes DOR | abs rel: 0.160 -> 0.125 | Helped |
| Add fine-stage regularization (ablation) | Monodepth2 coarse-stage model · Cityscapes WIR | abs rel: 0.145 -> 0.102 | Helped |
| Add fine-stage regularization (ablation) | Monodepth2 coarse-stage model · Cityscapes DOR | abs rel: 0.303 -> 0.119 | Helped |
| Replace cost-volume-weighted LREG with the filtering scheme in [40] (ablation) | Monodepth2 coarse-stage model · Cityscapes WIR | abs rel: 0.102 -> 0.124 | Hurt |
| Replace cost-volume-weighted LREG with the filtering scheme in [40] (ablation) | Monodepth2 coarse-stage model · Cityscapes DOR | abs rel: 0.119 -> 0.214 | Hurt |
| Use L0_REG instead of cost-volume-weighted LREG (ablation) | Monodepth2 coarse-stage model · Cityscapes WIR | abs rel: 0.102 -> 0.117 | Hurt |
Surprising findings
- Vanilla automasking fails to learn accurate object depth because it only masks objects moving with the camera's speed and direction.
- Filtering pixels with a depth-consistency threshold (Filtering [40]) yields weaker results than the cost-volume-weighted regularization.
Limitations
- Relies on pre-computed instance segmentation masks of dynamic classes during training, requiring an external segmentation model.
- Assumes dynamic objects contact the ground, which may not hold for all objects or heavily occluded bottoms.
- Gains on KITTI are smaller since it contains fewer moving objects.
Experimental setup
Datasets: Cityscapes KITTI COCO
Benchmarks: Cityscapes KITTI
Metrics: abs rel sq rel rmse logrmse a1 a2 a3
Baselines: Monodepth2 MonoViT HR-Depth CADepth DynamicDepth RM-Depth InstaDM Li et al. Lee et al. Petrovai et al. FSRE-Depth SGDepth
Builds on: Monodepth2 MonoViT edge-aware disparity smoothness loss Mask2Former
Closest papers by meaning
- Mining Supervision for Dynamic Regions in Self-Supervised Monocular Depth Estimation (CVPR 2024 · similarity 0.90)
- Improved Monocular Depth Prediction Using Distance Transform Over Pre-semantic Contours with Self-supervised Neural Networks (CVPR 2025 · similarity 0.88)
- GeoDepth: From Point-to-Depth to Plane-to-Depth Modeling for Self-Supervised Monocular Depth Estimation (CVPR 2025 · similarity 0.88)
- PPEA-Depth: Progressive Parameter-Efficient Adaptation for Self-Supervised Monocular Depth Estimation (AAAI 2024 · similarity 0.88)
- SQLdepth: Generalizable Self-Supervised Fine-Structured Monocular Depth Estimation (AAAI 2024 · similarity 0.87)
- Mind The Edge: Refining Depth Edges in Sparsely-Supervised Monocular Depth Estimation (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.