Research map / 3D vision / Depth and stereo
IBD-SLAM: Learning Image-Based Depth Fusion for Generalizable SLAM
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes IBD-SLAM, a generalizable NeRF-based visual SLAM that learns an image-based depth fusion model over xyz-maps so it runs on unseen RGBD videos with only light pose optimization.
Problem
Neural-field SLAM methods require lengthy scene-specific optimization for each scene, making mapping slow and preventing generalization to unseen scenes.
Approach
Adapts IBRNet-style image-based rendering to depth fusion: reference depth maps are back-projected into world-frame xyz-maps, and a learned network blends them via volume rendering to predict the target xyz-map, enabling zero-shot mapping with only pose optimization at test time.
Key results
- On Replica, IBD-SLAM reaches Depth L1 1.53, Accuracy 1.83, Completion 2.02, and PSNR 28.5, the best PSNR among compared methods.
- Mapping is about 10x faster than the prior state of the art, with 12.3 ms x 1 iteration per map step versus 82.5 ms x 60 for NICE-SLAM.
- Average ATE RMSE on ScanNet is 7.32 cm, the best among the compared methods (ESLAM 7.58, Co-SLAM 8.38).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use image-based xyz depth fusion for mapping | IBD-SLAM · Replica | Accuracy: 2.34 -> 1.83 | Helped |
| Use image-based xyz depth fusion for mapping | IBD-SLAM · Replica | PSNR: 27.9 -> 28.5 | Helped |
| Use IBD-SLAM tracking | IBD-SLAM · ScanNet | ATE RMSE [cm]: 7.58 -> 7.32 | Helped |
| Use feed-forward mapping rather than per-scene mapping optimization | IBD-SLAM · Replica | Mapping time [ms x it.]: 18.4x15 -> 12.3x1 | Helped |
| Exclude novel views during reconstruction (ablation) | IBD-SLAM · Replica room-2 | Depth L1: 1.53 -> 1.80 | Hurt |
| Use one shared network for RGB and xyz data (ablation) | IBD-SLAM · Replica room-2 | Depth L1: 1.53 -> 2.35 | Hurt |
| Remove Lxyz (ablation) | IBD-SLAM · Replica room-2 | Depth L1: 1.53 -> 3.02 | Hurt |
| Remove Ldepth (ablation) | IBD-SLAM · Replica room-2 | Depth L1: 1.53 -> 2.09 | Hurt |
| Remove Lrgb (ablation) | IBD-SLAM · Replica room-2 | Depth L1: 1.53 -> 2.17 | Hurt |
| Remove Lnormal (ablation) | IBD-SLAM · Replica room-2 | Depth L1: 1.53 -> 1.65 | Hurt |
| Remove Lwrap (ablation) | IBD-SLAM · Replica room-2 | Depth L1: 1.53 -> 1.62 | Hurt |
| Remove Lflow (ablation) | IBD-SLAM · Replica room-2 | Depth L1: 1.53 -> 1.57 | Hurt |
Limitations
- Tracking accuracy is comparable rather than uniformly superior (e.g., TUM avg 2.0 cm ties ESLAM).
- Reliance on monocular depth estimator and SuperPoint/SuperGlue correspondences for supervision and tracking.
- Mesh reconstruction relies on Poisson surface reconstruction from rendered point clouds.
Experimental setup
Datasets: DeepVoxels Google Scanned Objects Spaces Dataset Local Light Field Fusion IBRNet scans Replica TUM-RGBD ScanNet
Benchmarks: Replica TUM-RGBD ScanNet
Metrics: Depth L1 Accuracy Completion Completion Ratio ATE RMSE PSNR SSIM
Baselines: iMAP DI-Fusion NICE-SLAM ESLAM Co-SLAM Orb-SLAM2
Builds on: IBRNet NeRF NICE-SLAM SuperGlue Monocular depth estimator (Omnidata)
Closest papers by meaning
- Photo-SLAM: Real-time Simultaneous Localization and Photorealistic Mapping for Monocular Stereo and RGB-D Cameras (CVPR 2024 · similarity 0.88)
- VTGaussian-SLAM: RGBD SLAM for Large Scale Scenes with Splatting View-Tied 3D Gaussians (ICML 2025 · similarity 0.88)
- Loopy-SLAM: Dense Neural SLAM with Loop Closures (CVPR 2024 · similarity 0.88)
- GS-SLAM: Dense Visual SLAM with 3D Gaussian Splatting (CVPR 2024 · similarity 0.87)
- VGGT-SLAM: Dense RGB SLAM Optimized on the SL(4) Manifold (NeurIPS 2025 · similarity 0.86)
- Benchmarking Implicit Neural Representation and Geometric Rendering in Real-Time RGB-D SLAM (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.