Research map / 3D vision / Depth and stereo
Mining Supervision for Dynamic Regions in Self-Supervised Monocular Depth Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a self-supervised pipeline that decouples static and dynamic regions, uses a depth scale alignment module to produce scale-consistent pseudo depth labels for moving objects, and trains a monocular depth network on them.
Problem
Self-supervised monocular depth estimation from video performs poorly on dynamic (moving object) regions because of scale ambiguity between depth and object motion.
Approach
Decouple depth for static and dynamic regions: estimate scene depth and object motion unsupervised, estimate per-object depth, then use a Depth Scale Alignment network trained on static objects to rescale moving objects, producing scale-consistent pseudo depth labels used to supervise the final depth network.
Key results
- On Cityscapes, the method reduces dynamic-region Sq Rel by 52.6% relative to the best prior baseline (0.870 absolute margin).
- Applied to ResNet18, PackNet, DiffNet and BrNet, the method reduces dynamic-region Sq Rel by an average of 52.1%.
- On KITTI, the method achieves at least a 14.4% reduction in dynamic-region Sq Rel over prior works.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add pixel-wise object motion prediction (ablation) | ResNet18 depth estimation · Cityscapes dynamic region | Sq Rel: 11.169 -> 1.650 | Helped |
| Add scale-consistent pseudo-depth supervision (ablation) | ResNet18 with pixel motion · Cityscapes dynamic region | Sq Rel: 1.650 -> 0.784 | Helped |
| Add a pre-trained semantic segmentation prior (ablation) | ResNet18 with pseudo-depth supervision · Cityscapes dynamic region | Sq Rel: 0.784 -> 0.647 | Helped |
| Add pixel-wise object motion prediction (ablation) | PackNet depth estimation · Cityscapes dynamic region | Sq Rel: 2.667 -> 0.987 | Helped |
| Add scale-consistent pseudo-depth supervision (ablation) | PackNet with pixel motion · Cityscapes dynamic region | Sq Rel: 0.987 -> 0.703 | Helped |
| Add a pre-trained semantic segmentation prior (ablation) | PackNet with pseudo-depth supervision · Cityscapes dynamic region | Sq Rel: 0.703 -> 0.666 | Helped |
| Add pixel-wise object motion prediction (ablation) | DiffNet depth estimation · Cityscapes dynamic region | Sq Rel: 4.369 -> 1.646 | Helped |
| Add scale-consistent pseudo-depth supervision (ablation) | DiffNet with pixel motion · Cityscapes dynamic region | Sq Rel: 1.646 -> 0.657 | Helped |
| Add a pre-trained semantic segmentation prior (ablation) | DiffNet with pseudo-depth supervision · Cityscapes dynamic region | Sq Rel: 0.657 -> 0.692 | Hurt |
| Add pixel-wise object motion prediction (ablation) | BrNet depth estimation · Cityscapes dynamic region | Sq Rel: 3.440 -> 2.041 | Helped |
| Add scale-consistent pseudo-depth supervision (ablation) | BrNet with pixel motion · Cityscapes dynamic region | Sq Rel: 2.041 -> 0.670 | Helped |
| Add a pre-trained semantic segmentation prior (ablation) | BrNet with pseudo-depth supervision · Cityscapes dynamic region | Sq Rel: 0.670 -> 0.577 | Helped |
Surprising findings
- Joint pixel-wise motion estimation alone can worsen dynamic-region depth due to scale inconsistency, which pseudo labels fix.
- Self-training with the model's own predictions gives gains only at the first iteration and then plateaus.
Limitations
- Does not generalize well to indoor scenes where moving objects such as humans are non-rigid.
- KITTI has few moving objects, so gains in all-region metrics are marginal.
- Relies on self-supervised segmentation and motion masks whose errors can propagate into pseudo labels.
Experimental setup
Datasets: Cityscapes KITTI
Benchmarks: Cityscapes KITTI Eigen split
Metrics: Abs Rel Sq Rel delta < 1.25
Baselines: Monodepth2 Li et al. (pixel-wise motion) InstaDM RMDepth DaCNN PackNet DiffNet BrNet
Builds on: Monodepth2 Li et al. motion disentanglement InstaDM SEEM
Closest papers by meaning
- AdaDepth: Exploiting Inherent Scene Information for Self-Supervised Depth Estimation in Dynamic Scenes (AAAI 2026 · similarity 0.92)
- Learning Monocular Depth in Dynamic Scenes via Instance-Aware Projection Consistency (AAAI 2021 · similarity 0.91)
- The Temporal Opportunist: Self-Supervised Multi-Frame Monocular Depth (CVPR 2021 · similarity 0.91)
- From-Ground-To-Objects: Coarse-to-Fine Self-supervised Monocular Depth Estimation of Dynamic Objects with Ground Contact Prior (CVPR 2024 · similarity 0.90)
- Improved Monocular Depth Prediction Using Distance Transform Over Pre-semantic Contours with Self-supervised Neural Networks (CVPR 2025 · similarity 0.88)
- PPEA-Depth: Progressive Parameter-Efficient Adaptation for Self-Supervised Monocular Depth Estimation (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.