Research map / 3D vision / Depth and stereo
Evaluating Robustness of Monocular Depth Estimation with Procedural Scene Perturbations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces PDE, a procedurally generated benchmark of 3D scenes (Infinigen) with 12 controlled scene perturbations (camera, object, material, lighting, occlusion, deformation, OOD background) to measure accuracy stability and self-consistency of 9 monocular depth models.
Problem
Standard monocular depth benchmarks measure accuracy but not robustness to controlled changes in scene content such as camera, object, material, and lighting variations.
Approach
Use procedural generation (Infinigen) to create paired base and perturbed scenes that vary one factor at a time, and define accuracy stability (variance of error) and self-consistency (deviation from base prediction) metrics to evaluate depth models.
Key results
- DepthPro achieves the lowest average AbsRel error (1.24) across perturbations, about 13% lower than the next best model MoGe (1.46).
- UniDepthV2 has the best accuracy stability (0.27 average AbsRel instability) and best self-consistency among scale-invariant-compatible models.
- Camera dolly zoom and camera roll are among the most difficult perturbations, while lighting is the easiest (stability 0.18).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply camera roll perturbations | Depth estimation models · PDE object-level | AbsRel average error: 2.06 | Hurt |
| Apply object occlusion perturbations | Depth estimation models · PDE object-level | AbsRel average error: 1.97 | Hurt |
| Apply lighting perturbations | Depth estimation models · PDE object-level | AbsRel accuracy instability: 0.18 | No clear effect |
| Use DepthPro instead of other evaluated models | PDE object-level evaluation · Average across perturbations | AbsRel average error: DepthPro 1.24; UniDepthV2 1.40 | Helped |
| Use UniDepthV2 instead of other evaluated models | PDE object-level evaluation · Average across perturbations | AbsRel accuracy instability: UniDepthV2 0.27; DepthPro 0.30 | Helped |
| Use UniDepthV2 instead of other evaluated models | PDE object-level evaluation · Average across evaluated self-consistency perturbations | AbsRel self-consistency: UniDepthV2 1.40; DepthPro 1.52 | Helped |
| Apply object resizing perturbations | Depth estimation models · PDE object-level | AbsRel accuracy instability: 0.18 | No clear effect |
| Apply scene or object material swaps | Depth estimation models · PDE object-level | AbsRel average error: 1.71; 1.77 | Hurt |
| Apply non-rigid object deformation | Depth estimation models · PDE object-level | AbsRel average error: 1.68 | No clear effect |
Surprising findings
- Camera perturbations such as focal length change and image roll are among the most challenging, more so than object pose changes.
- DepthPro has the lowest error but is less stable than UniDepthV2 and Metric3D V2, showing accuracy and robustness do not coincide.
Limitations
- Only 5 object categories and 38 scenes; limited coverage of scene types such as urban or driving scenes.
- Perturbations are applied one at a time; combined perturbations are not evaluated.
- Infinigen renders have a domain gap from real images.
Experimental setup
Datasets: Infinigen Nature Infinigen Indoors
Benchmarks: PDE
Metrics: AbsRel Log10 delta1 delta0.125 RMSE accuracy stability self-consistency average error
Baselines: MiDaS DepthAnything DepthAnything V2 ZoeDepth UniDepthV2 Metric3D V2 Marigold DepthPro MoGe
Builds on: Infinigen Marigold RoboDepth
Closest papers by meaning
- Depth2Pose: A Pose-Based Benchmark for Monocular Depth Estimation without Ground-Truth Depth (NeurIPS 2026 · similarity 0.86)
- Depth Pro: Sharp Monocular Metric Depth in Less Than a Second (ICLR 2025 · similarity 0.86)
- MonoInstance: Enhancing Monocular Priors via Multi-view Instance Alignment for Neural Rendering and Reconstruction (CVPR 2025 · similarity 0.86)
- Scalable Autoregressive Monocular Depth Estimation (CVPR 2025 · similarity 0.86)
- Mono2Stereo: A Benchmark and Empirical Study for Stereo Conversion (CVPR 2025 · similarity 0.86)
- Depth Anything V2 (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.