Research map / 3D vision / Depth and stereo
Photo-SLAM: Real-time Simultaneous Localization and Photorealistic Mapping for Monocular Stereo and RGB-D Cameras
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Photo-SLAM builds a real-time SLAM system on a hyper primitives map (ORB features plus 3D Gaussians) with geometry-based densification and Gaussian-pyramid progressive training for photorealistic mapping on monocular, stereo, and RGB-D cameras.
Problem
Existing neural-rendering SLAM systems rely on implicit representations that are too computationally heavy for real-time use on portable devices, and they need depth information or bounded scenes to converge.
Approach
Decouple localization (ORB-SLAM3 factor-graph tracking on explicit geometric features) from photorealistic mapping (3D Gaussian splatting on a hyper primitives map), add geometry-based densification from inactive 2D keypoints, and train with a Gaussian-pyramid multi-resolution loss schedule.
Key results
- On Replica (RGB-D), Photo-SLAM reaches 34.958 PSNR and 1084 rendering FPS, versus 30.594 PSNR for ESLAM and 34.632 for Point-SLAM at much lower rendering speed.
- On Replica (monocular), Photo-SLAM reaches 33.302 PSNR and 911 rendering FPS, versus 16.311 PSNR for Nice-SLAM without depth supervision.
- Photo-SLAM runs at about 18 tracking FPS and about 95 rendering FPS on the Jetson AGX Orin in monocular Replica runs.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add geometry-based densification (ablation) | Photo-SLAM · Replica, monocular | PSNR: 31.274 -> 33.302 | Helped |
| add geometry-based densification (ablation) | Photo-SLAM · Replica, RGB-D | PSNR: 33.296 -> 34.958 | Helped |
| add Gaussian-Pyramid-based learning (ablation) | Photo-SLAM with geometry-based densification · Replica, monocular | PSNR: 20.002 -> 33.302 | Helped |
| add Gaussian-Pyramid-based learning (ablation) | Photo-SLAM with geometry-based densification · Replica, RGB-D | PSNR: 33.696 -> 34.958 | Helped |
| use Gaussian pyramid with n=1 instead of n=2 (ablation) | Photo-SLAM · Replica, monocular | PSNR: 33.302 -> 30.903 | Hurt |
| use Gaussian pyramid with n=1 instead of n=2 (ablation) | Photo-SLAM · Replica, RGB-D | PSNR: 34.958 -> 34.634 | Hurt |
| use Gaussian pyramid with n=3 instead of n=2 (ablation) | Photo-SLAM · Replica, monocular | PSNR: 33.302 -> 31.563 | Hurt |
| use Gaussian pyramid with n=3 instead of n=2 (ablation) | Photo-SLAM · Replica, RGB-D | PSNR: 34.958 -> 33.305 | Hurt |
Surprising findings
- Adding geometry-based densification without Gaussian-pyramid learning degraded monocular mapping quality and rendering speed, as densified primitives with inaccurate positions became encumbrances.
- A 4-level Gaussian pyramid performed worse than 3 levels, attributed possibly to overfitting low-level features during incremental mapping.
Limitations
- Monocular mapping quality is sensitive to depth estimated from nearest ORB neighbours, which can yield inaccurate primitive positions.
- Mesh reconstruction is explicitly out of scope of the work.
- Stereo results are reported on EuRoC with lower PSNR (about 14-17 dB) than RGB-D results.
Experimental setup
Datasets: Replica TUM RGB-D EuRoC MAV ZED 2 outdoor stereo captures
Benchmarks: Replica TUM RGB-D EuRoC MAV
Metrics: ATE RMSE ATE STD PSNR SSIM LPIPS tracking FPS rendering FPS GPU memory usage
Baselines: ORB-SLAM3 DROID-SLAM Nice-SLAM Orbeez-SLAM Go-SLAM ESLAM Co-SLAM Point-SLAM BundleFusion
Builds on: ORB-SLAM3 3D Gaussian splatting Droid-SLAM NICE-SLAM
Closest papers by meaning
- Benchmarking Implicit Neural Representation and Geometric Rendering in Real-Time RGB-D SLAM (CVPR 2024 · similarity 0.89)
- VTGaussian-SLAM: RGBD SLAM for Large Scale Scenes with Splatting View-Tied 3D Gaussians (ICML 2025 · similarity 0.88)
- IBD-SLAM: Learning Image-Based Depth Fusion for Generalizable SLAM (CVPR 2024 · similarity 0.88)
- GS-SLAM: Dense Visual SLAM with 3D Gaussian Splatting (CVPR 2024 · similarity 0.87)
- VGGT-SLAM: Dense RGB SLAM Optimized on the SL(4) Manifold (NeurIPS 2025 · similarity 0.87)
- 4DTAM: Non-Rigid Tracking and Mapping via Dynamic Surface Gaussians (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.