Research map / 3D vision / Depth and stereo
Intrinsic Image Decomposition for Robust Self-supervised Monocular Depth Estimation on Reflective Surfaces
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Adds a self-supervised intrinsic residual decomposition branch to monocular depth training so that reflective pixels (violating the Lambertian assumption) are masked out of the photometric loss, plus a pseudo-depth distillation stage.
Problem
Self-supervised monocular depth estimation relies on photometric consistency under a Lambertian assumption, which fails on reflective and non-Lambertian indoor surfaces and degrades depth accuracy there.
Approach
Jointly learn an intrinsic residual decomposition (diffuse + view-dependent residual) from unaligned dynamic-view monocular sequences, using depth-based view synthesis for cross-view diffuse consistency; then compare photometric error of RGB pairs vs. pseudo-diffuse pairs with a Mahalanobis-distance criterion to mask reflective pixels from depth training, and distill a fused pseudo-depth into a st
Key results
- On ScanNet-Reflection test, Monodepth2 + Ours reduces Abs Rel from 0.181 to 0.163 and MonoViT + Ours from 0.154 to 0.142.
- On the Booster dataset, Monodepth2 + Ours improves averaged metrics by 9.96%, HRDepth by 8.09%, and MonoViT by 4.39%.
- The distillation variant (Ours (distillation)) on ScanNet-Reflection validation improves Monodepth2 Abs Rel from 0.158 to 0.148 relative to the end-to-end model.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add intrinsic decomposition and reflection-aware depth training | Monodepth2, HRDepth, and MonoViT · ScanNet-Reflection Test set | Abs Rel: 0.181 -> 0.163; 0.182 -> 0.165; 0.154 -> 0.142 | Helped |
| Add intrinsic decomposition and reflection-aware depth training | Monodepth2, HRDepth, and MonoViT · ScanNet-Reflection Validation set | Abs Rel: 0.206 -> 0.158; 0.213 -> 0.160; 0.179 -> 0.139 | Helped |
| Add intrinsic decomposition and reflection-aware depth training | Monodepth2, HRDepth, and MonoViT · ScanNet-NoReflection Validation set | Abs Rel: 0.169 -> 0.174; 0.169 -> 0.169; 0.140 -> 0.144 | Mixed |
| Add intrinsic decomposition and reflection-aware depth training | Monodepth2, HRDepth, and MonoViT · 7-Scenes | Abs Rel: 0.210 -> 0.209; 0.193 -> 0.204; 0.173 -> 0.171 | Mixed |
| Add intrinsic decomposition and reflection-aware depth training | Monodepth2, HRDepth, and MonoViT · Booster | Abs Rel: 0.520 -> 0.471; 0.495 -> 0.462; 0.418 -> 0.396 | Helped |
| Add contrastive loss L_cts (ablation) | Monodepth2 + Ours · ScanNet-Reflection Test set | Abs Rel: 0.176 -> 0.163 | Helped |
| Add contrastive loss L_cts (ablation) | Monodepth2 + Ours · ScanNet-Reflection Test set | RMSE: 0.510 -> 0.472 | Helped |
Surprising findings
- Averaged over all metrics, Monodepth2 and MonoViT slightly degrade on the non-reflective ScanNet-NoReflection validation set (1.47% and 1.10%), while HRDepth slightly improves.
- On 7-Scenes, HRDepth + Ours degrades by about 2.15% on average across metrics.
Limitations
- Struggles with hard non-Lambertian cases such as transparent objects and fully reflective mirrors.
- Uses a 3-channel diffuse and grayscale residual for stability, which may miss chromatic specular effects.
- Does not address occlusion-related issues such as edge fattening.
Experimental setup
Datasets: ScanNetv2 7-Scenes Booster
Benchmarks: ScanNet-Reflection test ScanNet-Reflection validation ScanNet-NoReflection validation 7-Scenes Booster
Metrics: Abs Rel Sq Rel RMSE RMSE log delta < 1.25 delta < 1.25^2 delta < 1.25^3
Baselines: Monodepth2 HRDepth MonoViT Self-teaching 3D Distillation
Builds on: Monodepth2 MonoViT HRDepth intrinsic residual model 3D Distillation
Closest papers by meaning
- Self-supervised Monocular Depth Estimation Robust to Reflective Surface Leveraged by Triplet Mining (ICLR 2025 · similarity 0.91)
- Exploiting Pseudo Labels in a Self-Supervised Learning Framework for Improved Monocular Depth Estimation (CVPR 2022 · similarity 0.88)
- Revisiting Deep Intrinsic Image Decompositions (CVPR 2018 · similarity 0.87)
- On the Uncertainty of Self-Supervised Monocular Depth Estimation (CVPR 2020 · similarity 0.87)
- Improved Monocular Depth Prediction Using Distance Transform Over Pre-semantic Contours with Self-supervised Neural Networks (CVPR 2025 · similarity 0.87)
- DME: Unveiling the Bias for Better Generalized Monocular Depth Estimation (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.