Research map / 3D vision / Gaussian splatting
Scalable Benchmarking and Robust Learning for Noise-Free Ego-Motion and 3D Reconstruction from Noisy Video
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds a scalable noisy RGB-D data synthesis pipeline, the Robust-Ego3D benchmark (124 perturbation settings, 1,000 videos), and CorrGS, a correspondence-guided Gaussian Splatting SLAM method that improves pose tracking and appearance restoration under noise.
Problem
Dense neural SLAM and 3D reconstruction methods are evaluated and tuned on noise-free data, so their accuracy collapses under real-world perturbations such as motion deviations, imaging corruptions, depth noise, fast motion, and RGB-D desynchronization.
Approach
Synthesize controllable RGB-D perturbations from clean 3D meshes to benchmark dense neural SLAM, then use correspondence-initialized pose estimation and online appearance restoration against a noise-free Gaussian map to make tracking and mapping robust.
Key results
- On Robust-Ego3D, GO-SLAM (RGB-D) ATE rises from 0.46 cm to 3.78 cm under depth noise, performing worse than its monocular version.
- CARL raises CorrGS tracking success under partial darkening from 1/8 (baseline) to 8/8 with ATE 0.54 cm, close to the noise-free 0.45 cm, and restored RGB PSNR of 35.48 dB.
- CorrGS and CorrGS-L achieve the lowest ATE across all sparse-view fast-motion sequences, outperforming SplaTAM, SplaTAM-L, and Hero-SLAM.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add correspondence-guided pose learning (CPL) | CorrGS · Synthetic sparse-view videos with fast motion | ATE | Helped |
| Remove pose quality verification (PQV) (ablation) | CorrGS-L · Synthetic sparse-view videos with fast motion | ATE | Hurt |
| Add CARL for pose tracking and mapping (CARL-T&M) | CorrGS · Partial degraded illumination | Tracking success: 1/8 -> 8/8 | Helped |
| Omit the restored image in mapping (CARL-T) (ablation) | CARL · Partial degraded illumination | Tracking success: 8/8 -> 4/8 | Hurt |
| Use CARL solely for mapping (CARL-M), leaving pose optimization on unrestored RGB-D (ablation) | CARL · O-1 noisy illumination | PSNR: 12.67 dB -> 24.53 dB | Helped |
| Use CARL solely for mapping (CARL-M), without restoration for pose optimization (ablation) | CARL · Partial degraded illumination | Tracking success | Hurt |
| Remove the linear motion assumption (ablation) | SplaTAM · Original-speed clean videos | ATE (cm), average: 0.36 -> 5.68 | Hurt |
| Remove the linear motion assumption and increase pose optimization to 200 iterations (ablation) | SplaTAM · Original-speed clean videos | ATE (cm), average: 5.68 -> 15.30 | Hurt |
| Add noisy depth to the mixed perturbation setting | SplaTAM-S · Office-0 mixed perturbations | ATE (m): +0.204 | Hurt |
| Add RGB JPEG compression to the mixed perturbation setting | GO-SLAM · Office-0 mixed perturbations | ATE (m): +0.041 | No clear effect |
| Use RGB-D instead of monocular input | GO-SLAM · RGB imaging perturbations | ATE | Helped |
Surprising findings
- Trajectory accuracy does not always correlate with 3D reconstruction quality; Nice-SLAM tracks well under shot noise but color reconstruction degrades severely.
- Neural models show a local denoising property, e.g. iMAP produces nearly spatter-free RGB images, and depth missing causes minimal degradation.
Limitations
- CorrGS pilot studies only cover illumination changes; other perturbations are left for future work.
- Evaluation is on sparse-view synthetic video and a single real-world pilot sequence, with limited scope.
- Benchmark is indoor-only (Replica-style scenes); outdoor and cross-domain settings are not covered.
Experimental setup
Datasets: Replica TUM-RGBD TartanAir
Benchmarks: Robust-Ego3D Replica TUM-RGBD
Metrics: ATE PSNR depth L1 loss
Baselines: iMAP Nice-SLAM CO-SLAM GO-SLAM SplaTAM ORB-SLAM3 Hero-SLAM
Builds on: 3D Gaussian Splatting SplaTAM Nice-SLAM GO-SLAM ORB-SLAM3
Closest papers by meaning
- MAC-Ego3D: Multi-Agent Gaussian Consensus for Real-Time Collaborative Ego-Motion and Photorealistic 3D Reconstruction (CVPR 2025 · similarity 0.88)
- Layered Motion Fusion: Lifting Motion Segmentation to 3D in Egocentric Videos (CVPR 2025 · similarity 0.87)
- HiCoM: Hierarchical Coherent Motion for Dynamic Streamable Scenes with 3D Gaussian Splatting (NeurIPS 2024 · similarity 0.86)
- Stereo4D: Learning How Things Move in 3D from Internet Stereo Videos (CVPR 2025 · similarity 0.86)
- E-MoFlow: Learning Egomotion and Optical Flow from Event Data via Implicit Regularization (NeurIPS 2025 · similarity 0.86)
- PF3plat: Pose-Free Feed-Forward 3D Gaussian Splatting for Novel View Synthesis (ICML 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.