Research map / 3D vision / Depth and stereo
PatchFusion: An End-to-End Tile-Based Framework for High-Resolution Monocular Metric Depth Estimation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PatchFusion, an end-to-end tile-based framework that fuses a coarse global ZoeDepth prediction with fine patch predictions via a guided fusion network, G2L module, and consistency-aware training/inference, for high-resolution metric depth estimation.
Problem
Monocular metric depth estimation models are limited to low input resolutions (e.g., 384x512), while modern cameras capture multi-megapixel images; existing high-resolution strategies suffer from error propagation, lost high-frequency detail, scale inconsistency across patches, and reliance on post-
Approach
A guided fusion U-Net takes coarse global depth, fine tile depth, and image tiles, with high-level features from a Global-to-Local Swin-transformer module injected into skip connections and decoder, and a consistency loss on overlapping patch features and depths, enabling stitching without post-processing.
Key results
- On UnrealStereo4K, PatchFusion built on ZoeDepth improves RMSE by 17.3% and REL by 14.7% over ZoeDepth (RMS 1.0655 vs 1.2887 coarse baseline with PF R=128 best).
- On MVS-Synth, RMS improves by about 29.4% and SiLog by 30.8% over ZoeDepth.
- Consistency-aware training and inference reduce consistency error substantially (e.g., CE 0.1441 to 0.0464 with CAI on P=49 in ablation).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply PatchFusion to ZoeDepth | ZoeDepth · UnrealStereo4K | RMS: 1.2887 -> 1.0655 | Helped |
| Apply PatchFusion to ZoeDepth | ZoeDepth · MVS-Synth | RMS: 1.2676 -> 0.8944 | Helped |
| Add feature guidance (ablation) | Guided Fusion Network · UnrealStereo4K | REL: 0.0450 -> 0.0423 | Helped |
| Add the G2L module (ablation) | Guided Fusion Network with feature guidance · UnrealStereo4K | CE: 0.2318 -> 0.1714 | Helped |
| Add consistency-aware training on predictions (ablation) | Fusion with feature guidance and G2L · UnrealStereo4K | REL: 0.0414 -> 0.0411 | Helped |
| Apply consistency-aware training to intermediate features (ablation) | Fusion with feature guidance · UnrealStereo4K | REL: 0.0423 -> 0.0403 | Helped |
| Apply consistency constraints to both predictions and intermediate features (ablation) | PatchFusion fusion network · UnrealStereo4K | REL: 0.0403 -> 0.0399 | Helped |
| Use 49 patches without consistency-aware inference (ablation) | PatchFusion · UnrealStereo4K | REL: 0.0399 -> 0.0398 | No clear effect |
| Add consistency-aware inference with additional patches (ablation) | PatchFusion · UnrealStereo4K | REL: 0.0398 -> 0.0392 | Helped |
| Add consistency-aware inference with additional patches (ablation) | PatchFusion · UnrealStereo4K | SEE: 0.8397 -> 0.8462 | Hurt |
Surprising findings
- Directly feeding global guidance features from the coarse network without the G2L module still suffers scale-shift issues, and the standalone fusion network barely improves over the coarse baseline.
Limitations
- Computational cost grows with number of patches used at inference.
- Lack of high-resolution real-world training data; synthetic-to-real transfer leaves scale errors.
- UnrealStereo4K has few images with large homogeneous foreground objects.
Experimental setup
Datasets: UnrealStereo4K MVS-Synth NYU-v2 Middlebury 2014
Benchmarks: UnrealStereo4K MVS-Synth Middlebury 2014
Metrics: delta1 REL RMS SiLog SEE CE
Baselines: ZoeDepth iDisc BoostingDepth Graph-GDSR SMD-Net
Builds on: ZoeDepth BoostingDepth SMD-Net ControlNet
Closest papers by meaning
- Depth Pro: Sharp Monocular Metric Depth in Less Than a Second (ICLR 2025 · similarity 0.89)
- DepthMaster: Unified Monocular Depth Estimation for Perspective and Panoramic Images (NeurIPS 2026 · similarity 0.88)
- A Simple yet Universal Framework for Depth Completion (NeurIPS 2024 · similarity 0.88)
- SharpDepth: Sharpening Metric Depth Predictions Using Diffusion Distillation (CVPR 2025 · similarity 0.88)
- Depth2Pose: A Pose-Based Benchmark for Monocular Depth Estimation without Ground-Truth Depth (NeurIPS 2026 · similarity 0.87)
- DepthFM: Fast Generative Monocular Depth Estimation with Flow Matching (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.