Research map / 3D vision / Depth and stereo
Adaptive Fusion of Single-View and Multi-View Depth for Autonomous Driving
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AFNet, which adaptively fuses a single-view monocular depth branch and a multi-view cost-volume branch using a warping-based confidence map, and introduces a noisy-pose robustness benchmark for depth estimation on DDAD and KITTI.
Problem
Multi-view depth estimation degrades severely when camera poses are noisy, as is common in autonomous driving, and existing single/multi-view fusion methods assume ideal poses and can fall below single-view accuracy.
Approach
A two-branch network (single-view and multi-view) whose outputs are fused by an adaptive fusion module guided by three confidence maps, including a warping confidence map computed from source images warped with the predicted multi-view depth and the given poses, so the system selects the more reliable branch per pixel.
Key results
- On KITTI Eigen, AFNet achieves 0.039 AbsRel, reported as a 27.8% reduction over MaGNet (0.054).
- On DDAD, AFNet achieves 0.088 AbsRel and 1.41 SqRel, over 15% better AbsRel than IterMVS and MaGNet.
- Under noisy pose settings on DDAD, AFNet attains the best R-Rel (0.168 at δ=0.05 vs 0.195 for MaGNet), remaining above single-view baseline levels.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fuse single-view and multi-view depth with the adaptive fusion module (ablation) | AFNet · DDAD | SqRel: 1.59 -> 1.55 | Helped |
| Directly fuse single-view and multi-view branch outputs with convolution (ablation) | Base-RF · DDAD | SqRel: 1.78 -> 1.99 | Hurt |
| Add single-view depth features to the multi-view branch (ablation) | Base-FF · DDAD | SqRel: 1.78 -> 1.59 | Helped |
| Share the feature-extraction network parameters between branches (ablation) | BASE-PS · DDAD | SqRel: 1.55 -> 1.49 | Helped |
| Replace Epipolar Attention Module with cost volume (ablation) | AFNet · DDAD | SqRel: 1.49 -> 1.41 | Helped |
| Use adaptive fusion of single-view and multi-view predictions | AFNet · DDAD dynamic object region | SqRel: 1.879 -> 1.484 | Helped |
| Adaptively fuse single-view and multi-view depth rather than use the multi-view branch alone | AFNet · DDAD pose noise δ = 0.05 | AbsRel: 0.175 -> 0.164 | Helped |
| Use AFNet instead of MaGNet under noisy input poses | DDAD depth estimation · DDAD pose noise robustness aggregate | R-Rel: 0.195 -> 0.168 | Helped |
| Use AFNet instead of IterMVS | DDAD depth estimation · DDAD | AbsRel: 0.104∗ -> 0.088 | Helped |
| Use AFNet instead of Metric3D | KITTI Eigen split depth estimation · KITTI Eigen split | AbsRel: 0.053 -> 0.039 | Helped |
| Test DDAD-trained models on ScanNet without retraining | AFNet versus MVS2D · ScanNet zero-shot | AbsRel: 0.098 -> 0.091 | Helped |
Surprising findings
- Existing single/multi-view fusion methods such as MaGNet and MVS2D can perform worse than single-view methods under noisy poses because the multi-view branch injects wrong guidance.
- Naive convolutional fusion of single-view and multi-view outputs reduced accuracy relative to the multi-view branch alone in the ablation.
Limitations
- Robustness evaluation relies on synthetically perturbed poses plus a single SLAM system (ORB-SLAM2) on KITTI Odometry.
- Multi-view branch still depends on pose quality; noise was added during training for all compared methods.
- Monocular ORB-SLAM2 sometimes fails, so evaluation covers only sequence portions before crash.
Experimental setup
Datasets: DDAD KITTI ScanNet
Benchmarks: DDAD KITTI Eigen split ScanNet KITTI Odometry (SLAM poses)
Metrics: AbsRel SqRel RMSE R-Rel (mean + std under noisy poses)
Baselines: Monodepth2 FeatDepth DORN BTS AdaBins Metric3D PMNet Deepv2d CasMVS MVSNet IterMVS MVS2D
Builds on: MVS2D MaGNet IterMVS ConvNeXt
Closest papers by meaning
- Depth2Pose: A Pose-Based Benchmark for Monocular Depth Estimation without Ground-Truth Depth (NeurIPS 2026 · similarity 0.86)
- A Global Depth-Range-Free Multi-View Stereo Transformer Network with Pose Embedding (NeurIPS 2024 · similarity 0.86)
- STViT: Improving Self-Supervised Multi-Camera Depth Estimation with Spatial-Temporal Context and Adversarial Geometry Regularization (Student Abstract) (AAAI 2024 · similarity 0.86)
- Depth-Guided Robust and Fast Point Cloud Fusion NeRF for Sparse Input Views (AAAI 2024 · similarity 0.86)
- FusionFormer: A Concise Unified Feature Fusion Transformer for 3D Pose Estimation (AAAI 2024 · similarity 0.86)
- MVSAnywhere: Zero-Shot Multi-View Stereo (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.