Research map / 3D vision / Depth and stereo
SS3DM: Benchmarking Street-View Surface Reconstruction with a Synthetic 3D Mesh Dataset
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces SS3DM, a synthetic CARLA-based street-view dataset with precise ground-truth triangle meshes (with normals), multi-camera/multi-LiDAR sequences, and benchmarks six surface reconstruction methods using IoU, F-score, CD, and normal CD.
Problem
Existing street-view datasets (KITTI, Waymo, nuScenes) provide only noisy LiDAR points as geometric ground truth, which lack precision for evaluating surface positions and provide no surface normals for evaluating normal quality of reconstructed surfaces.
Approach
Export precise ground-truth triangle meshes with normals from CARLA's Unreal Engine via a plugin, simulate a 6-camera, 5-LiDAR driving rig to produce realistic multi-view inputs, and benchmark surface reconstruction with position and normal-aware metrics (CD + CD_N).
Key results
- StreetSurf (Full) achieves the best mean CD + CD_N of 1.524 across all sequences, versus 1.962 for NeRF-LOAM and 3.252 for R3D3.
- On long sequences, no evaluated method reaches an F-score above 0.1 (best F-score 0.068).
- Distance metrics F-score and CD do not reflect visual quality, while CD_N correlates better with qualitative results.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add RGB input to the LiDAR-only StreetSurf setup (ablation) | StreetSurf · SS3DM mean | CD + CD_N: 1.701 -> 1.524 | Helped |
| Add LiDAR input to the RGB-only StreetSurf setup (ablation) | StreetSurf · SS3DM mean | CD + CD_N: 1.995 -> 1.524 | Helped |
| Use LiDAR-only rather than RGB+LiDAR inputs (ablation) | StreetSurf · SS3DM mean | F-score: 0.198 -> 0.215 | Helped |
| Use RGB+LiDAR rather than LiDAR-only inputs (ablation) | StreetSurf · SS3DM mean | CD + CD_N: 1.701 -> 1.524 | Helped |
Surprising findings
- NeRF-LOAM achieves the best mean CD (0.551) and StreetSurf (LiDAR) the best F-score, yet both produce worse surfaces than StreetSurf (Full) visually.
- SuGaR, which performs well on object-level scenes, fails on large-scale outdoor scenes with bubble-like road artifacts and sky floaters.
Limitations
- Limited scene diversity (eight CARLA towns).
- No support for dynamic object reconstruction such as moving cars and pedestrians.
- Only part of the dataset is currently released; full release is promised later.
Experimental setup
Datasets: SS3DM
Benchmarks: SS3DM
Metrics: IoU F-score Chamfer Distance Normal Chamfer Distance Precision Recall Accuracy Completeness
Baselines: R3D3 NeRF-LOAM UrbanNeRF SuGaR StreetSurf (RGB) StreetSurf (LiDAR)
Builds on: CARLA StreetSurf UrbanNeRF NeRF-LOAM SuGaR
Closest papers by meaning
- ViiNeuS: Volumetric Initialization for Implicit Neural Surface Reconstruction of Urban Scenes with Limited Image Overlap (CVPR 2025 · similarity 0.86)
- USCILab3D: A Large-scale, Long-term, Semantically Annotated Outdoor Dataset (NeurIPS 2024 · similarity 0.86)
- LiSu: A Dataset and Method for LiDAR Surface Normal Estimation (CVPR 2025 · similarity 0.86)
- Effective Multi-sensor Conditioning for Street-view Novel-view Synthesis (NeurIPS 2026 · similarity 0.85)
- LidaRF: Delving into Lidar for Neural Radiance Field on Street Scenes (CVPR 2024 · similarity 0.85)
- D$^2$GS: Dense Depth Regularization for LiDAR-free Urban Scene Reconstruction (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.