Research map / 3D vision / Depth and stereo
Benchmarking Implicit Neural Representation and Geometric Rendering in Real-Time RGB-D SLAM
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds an open-source benchmark that unifies the SLAM pipeline and evaluates implicit neural representations (MLP, dense/sparse grid, tri-plane, factorization) with SDF-based geometric rendering functions for RGB-D SLAM, and proposes a hybrid encoding and an explicit hybrid encoding.
Problem
Existing NeRF-based RGB-D SLAM methods differ in many components beyond the implicit representation and rendering function, so there is no fair way to compare how INRs and geometric rendering functions affect mapping and localization.
Approach
A unified SLAM protocol (fixed sampling, keyframe selection, training) that isolates the INR F and geometric rendering G, with a leaderboard over Replica (lab) and NeuralRGBD (practical) scenarios; based on findings, new hybrid encoding (dense grid plus tri-plane) and explicit hybrid encoding (octree SDF prior with dense grid residual) are proposed.
Key results
- On Replica lab leaderboard, dense grid with SDF(density) rendering achieves ATE 1.31 cm and PSNR 27.77 dB, with the fastest mapping among top configurations.
- Hybrid encoding (dense grid plus tri-plane) on NeuralRGBD practical scenario reaches ATE 1.86 cm and Acc 2.40 cm with SDF(direct), improving on Co-SLAM's Acc 2.95 cm.
- Explicit hybrid encoding runs map updating at about 1 Hz versus about 0.25 Hz for NICE-SLAM on ScanNet scene0169.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use dense-grid encoding instead of other tested INRs | NeRF-SLAM with SDF (direct) · Replica lab scenario | ATE [cm]: 1.42 -> 1.37 | Helped |
| use hybrid dense-grid and tri-plane encoding | NeRF-SLAM · NeuralRGBD practical scenario | ATE [cm]: 2.29 -> 1.86 | Helped |
| use hybrid dense-grid and tri-plane encoding | NeRF-SLAM · NeuralRGBD practical scenario | PSNR [db]: +0.02 dB | Helped |
| use SDF (surface) rendering instead of SDF (direct) | NeRF-SLAM with dense-grid encoding · Replica lab scenario | ATE [cm]: 1.37 -> 135.39 | Hurt |
| use coupled geometry and appearance features rather than decoupled features (ablation) | MLP · Replica Room0 initialization | Depth L1 [cm]: 2.26 -> 1.68 | Helped |
| use coupled geometry and appearance features rather than decoupled features (ablation) | Dense grid · Replica Room0 initialization | PSNR [db]: 31.12 -> 29.66 | Hurt |
| use coupled geometry and appearance features rather than decoupled features (ablation) | Sparse grid · Replica Room0 initialization | PSNR [db]: 30.42 -> 28.45 | Hurt |
| use decoupled color encoding rather than coupled geometry and appearance features (ablation) | Tri-plane · Replica Room0 initialization | Depth L1 [cm]: 1.16 -> 1.08 | Helped |
| use decoupled color encoding rather than coupled geometry and appearance features (ablation) | Factorization · Replica Room0 initialization | PSNR [db]: 25.58 -> 28.36 | Helped |
| use decoupled color encoding rather than coupled geometry and appearance features (ablation) | MLP · Replica Room0 initialization | PSNR [db]: 23.76 -> 6.37 | Hurt |
| use SDF (surface) rendering rather than SDF (direct) (ablation) | Dense-grid NeRF-SLAM · Replica lab scenario | ATE [cm]: 1.37 -> 135.39 | Hurt |
| use SDF (density) rendering rather than SDF (direct) (ablation) | Dense-grid NeRF-SLAM · Replica lab scenario | Depth L1 [cm]: 1.50 -> 4.42 | Hurt |
Surprising findings
- Coupled geometry-color encoding helps MLP, dense and sparse representations, while decoupled encoding helps tri-plane and factorization.
- The neural implicit surfaces rendering (SDF(surface)) performs significantly worse than simpler SDF-based volume rendering in the online SLAM setting.
Limitations
- Results are tied to specific scenes; the paper does not declare a universally superior representation.
- Feature dimensions and resolution levels are fixed to small values for fair comparison, which may not reflect optimal real-world settings.
- Practical scenario excludes speed metrics due to variable sequence lengths.
Experimental setup
Datasets: Replica NeuralRGBD ScanNet
Benchmarks: Replica NeuralRGBD ScanNet
Metrics: Accuracy (cm) Completion (cm) Completion ratio (%) ATE RMSE (cm) PSNR (dB) Depth L1 (cm) tracking time mapping time
Baselines: NICE-SLAM Co-SLAM iMAP Neural-RGBD
Builds on: NeRF NICE-SLAM Co-SLAM Neural-RGBD Neural Implicit Surfaces
Closest papers by meaning
- Photo-SLAM: Real-time Simultaneous Localization and Photorealistic Mapping for Monocular Stereo and RGB-D Cameras (CVPR 2024 · similarity 0.89)
- Loopy-SLAM: Dense Neural SLAM with Loop Closures (CVPR 2024 · similarity 0.89)
- SLAM3R: Real-Time Dense Scene Reconstruction from Monocular RGB Videos (CVPR 2025 · similarity 0.88)
- Implicit Event-RGBD Neural SLAM (CVPR 2024 · similarity 0.86)
- Query Quantized Neural SLAM (AAAI 2025 · similarity 0.86)
- GS-SLAM: Dense Visual SLAM with 3D Gaussian Splatting (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.