Research map / 3D vision / Depth and stereo
Self-supervised Monocular Depth Estimation Robust to Reflective Surface Leveraged by Triplet Mining
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a reflection-aware triplet mining loss and a reflection-aware distillation scheme that improve self-supervised monocular depth estimation on reflective surfaces across Monodepth2, HRDepth, and MonoViT.
Problem
Self-supervised monocular depth estimation struggles on reflective (non-Lambertian) surfaces, where the photometric constancy assumption is violated, producing black-hole artifacts and wrong depth.
Approach
Use cross-view synthesized images to flag reflective pixels (where the cross-view photometric error is abnormally low) and apply a hinge triplet loss that pushes the same-view error below the different-view error only in those pixels; then distill from a triplet-trained teacher and a photometric-only teacher, fusing them via the reflective mask.
Key results
- On ScanNet-Reflection test, Ours (triplet loss only) with MonoViT reduces Abs Rel from 0.154 to 0.136 versus the self-supervised baseline, and the distilled variant reaches 0.126, matching 3D Distillation (0.127) without mesh rendering or ensembles.
- On Booster zero-shot, the distilled variant with Monodepth2 reduces Abs Rel from 0.520 to 0.419 relative to self-supervised training.
- On ScanNet-NoReflection validation, the triplet-only variant is comparable to self-supervised baselines (e.g., Monodepth2 Abs Rel 0.168 vs 0.169), while the distilled variant is comparable to 3D Distillation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add reflection-aware triplet mining loss | Monodepth2 · ScanNet-Reflection Test | Abs Rel: 0.181 -> 0.157 | Helped |
| add reflection-aware knowledge distillation to triplet mining | Monodepth2 · ScanNet-Reflection Validation | Abs Rel: 0.166 -> 0.151 | Helped |
| add reflection-aware triplet mining loss | Monodepth2 · ScanNet-NoReflection Validation | Abs Rel: 0.169 -> 0.168 | Helped |
| add reflection-aware knowledge distillation | Monodepth2 · NYU-v2 | Abs Rel: 0.171 -> 0.155 | Helped |
| add reflection-aware triplet mining loss | Monodepth2 · KITTI | Abs Rel: 0.118 -> 0.118 | No clear effect |
| add reflection-aware triplet mining loss | Monodepth2 · 7-Scenes | Abs Rel: 0.210 -> 0.207 | No clear effect |
| add reflection-aware knowledge distillation | Monodepth2 · Booster | Abs Rel: 0.520 -> 0.419 | Helped |
| apply triplet loss to all image regions instead of selectively using the reflective-region mask (ablation) | Monodepth2 · ScanNet-NoReflection Validation | Abs Rel: 0.169 -> 0.171 | Hurt |
| apply triplet loss to all image regions instead of selectively using the reflective-region mask (ablation) | Monodepth2 · ScanNet-Reflection Validation | Abs Rel: 0.206 -> 0.170 | Helped |
| selectively apply triplet loss to reflective regions rather than all regions (ablation) | Monodepth2 · ScanNet-NoReflection Validation | Abs Rel: 0.171 -> 0.168 | Helped |
| add reflection-aware triplet mining loss | RA-Depth · ScanNet-Reflection Test | Abs Rel: 0.161 -> 0.138 | Helped |
| add reflection-aware knowledge distillation after triplet mining | RA-Depth · ScanNet-Reflection Test | Abs Rel: 0.138 -> 0.130 | Helped |
Surprising findings
- Applying the triplet loss to all pixels (M_r = 1) degrades non-reflective performance notably, showing that selective masking is necessary.
- The triplet-only model slightly loses high-frequency detail on non-reflective surfaces, leading to no significant gain over self-supervised on 7-Scenes.
Limitations
- Cannot handle transparent or mirror (ToM) objects.
- Assumes ground-truth camera poses during training, as in 3D Distillation (for indoor setups).
- Fails when surfaces have multiple reflection lobes that violate the Lambertian-based assumption.
Experimental setup
Datasets: ScanNet-Reflection KITTI NYU-v2 7-Scenes Booster ScanNet-Original ScanNet-Robust
Benchmarks: ScanNet-Reflection ScanNet-NoReflection KITTI NYU-v2 7-Scenes Booster ScanNet-Original ScanNet-Robust
Metrics: Abs Rel Sq Rel RMSE RMSE log delta < 1.25
Baselines: Self-supervised (end-to-end) Self-Teaching 3D Distillation HRDepth MonoViT RA-Depth GasMono Lite-mono
Builds on: Monodepth2 3D Distillation triplet loss MonoViT HRDepth
Closest papers by meaning
- Intrinsic Image Decomposition for Robust Self-supervised Monocular Depth Estimation on Reflective Surfaces (AAAI 2025 · similarity 0.91)
- P3Depth: Monocular Depth Estimation With a Piecewise Planarity Prior (CVPR 2022 · similarity 0.88)
- DME: Unveiling the Bias for Better Generalized Monocular Depth Estimation (AAAI 2024 · similarity 0.87)
- A Simple yet Universal Framework for Depth Completion (NeurIPS 2024 · similarity 0.86)
- Improved Monocular Depth Prediction Using Distance Transform Over Pre-semantic Contours with Self-supervised Neural Networks (CVPR 2025 · similarity 0.86)
- Depth Anything: Unleashing the Power of Large-Scale Unlabeled Data (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.