Research map / 3D vision / Face and shape
ViiNeuS: Volumetric Initialization for Implicit Neural Surface Reconstruction of Urban Scenes with Limited Image Overlap
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ViiNeuS, a hybrid density-plus-SDF neural implicit method that progressively transitions from volumetric to surface rendering to reconstruct large urban driving scenes from limited-overlap images without LiDAR or geometric priors.
Problem
Neural implicit surface reconstruction of large driving scenes with limited image overlap is hard; existing methods need LiDAR, strong geometric priors, or long training times.
Approach
A hybrid architecture with separate density and SDF MLPs, where probabilistic density estimation (Mip-NeRF 360 proposal networks) selects near-surface samples, and alpha compositing transitions progressively from volumetric to SDF-based alpha, with probability-based sample attribution and SDF-gradient normalization for stability.
Key results
- Mean P→M error of 0.24 m for ViiNeuS versus 0.47 m for StreetSurf averaged across the four datasets.
- Single-scene training takes about 20 minutes versus 40 minutes for StreetSurf on the same GPU (RTX4090-equivalent), with 8 GB GPU memory versus 19 GB.
- On KITTI-360 ViiNeuS is on par with StreetSurf (P→M 0.12 m mean), and it outperforms explicit MVS methods (COLMAP, OpenMVS) on more complex datasets with smaller overlap.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use ViiNeuS instead of StreetSurf | Urban scene reconstruction · Four driving datasets, mean | P→M distance: 0.47m -> 0.24m | Helped |
| use ViiNeuS instead of StreetSurf | Urban scene reconstruction · Four driving datasets, mean | precision: 0.42% -> 0.46% | Helped |
| use ViiNeuS instead of StreetSurf | Urban scene reconstruction · Single scene | training time: 40 min -> 20 min | Helped |
| add progressive volumetric-to-surface SDF initialization (ablation) | ViiNeuS · KITTI-360 Seq. 31 | P→M distance: 0.35 -> 0.15 | Helped |
| add progressive volumetric-to-surface SDF initialization (ablation) | ViiNeuS · Pandaset Seq. 23 | P→M distance: 0.85 -> 0.51 | Helped |
| add progressive volumetric-to-surface SDF initialization (ablation) | ViiNeuS · nuScenes Seq. 0916 | P→M distance: 0.35 -> 0.25 | Helped |
| add progressive volumetric-to-surface SDF initialization (ablation) | ViiNeuS · Waymo Seq. 10061 | P→M distance: 0.3 -> 0.23 | Helped |
| supervise only the closest surface sample instead of accumulated normals (ablation) | ViiNeuS · KITTI-360 Seq. 31 | P→M distance: 0.19 -> 0.15 | Helped |
| supervise only the closest surface sample instead of accumulated normals (ablation) | ViiNeuS · Pandaset Seq. 23 | P→M distance: 0.80 -> 0.51 | Helped |
| supervise only the closest surface sample instead of accumulated normals (ablation) | ViiNeuS · nuScenes Seq. 0916 | P→M distance: 0.4 -> 0.25 | Helped |
| supervise only the closest surface sample instead of accumulated normals (ablation) | ViiNeuS · Waymo Seq. 10061 | P→M distance: 0.22 -> 0.23 | No clear effect |
| add the SDF regularization loss Ls (ablation) | ViiNeuS · KITTI-360 Seq. 31 | P→M distance: 0.24 -> 0.15 | Helped |
Surprising findings
- Neuralangelo could not be made to work on the driving sequences despite extensive tuning, because its spherical SDF initialization does not suit open driving scenes.
- On KITTI-360, explicit MVS methods (OpenMVS, COLMAP) achieve better precision than ViiNeuS, which the authors attribute to the high-overlap, large-FOV nature of that dataset.
Limitations
- Fails to reconstruct fine details in very wide and open driving sequences.
- StreetSurf-style road-surface initialization and Neuralangelo sphere initialization are not applicable, and ViiNeuS evaluation is limited to four static scenes per dataset.
Experimental setup
Datasets: KITTI-360 Pandaset Waymo Open Dataset nuScenes
Benchmarks: KITTI-360 Pandaset Waymo Open Dataset nuScenes
Metrics: Point-to-Mesh distance (m) Precision at 0.15m training time GPU memory parameter size
Baselines: StreetSurf COLMAP OpenMVS GOF Neuralangelo
Builds on: NeuS Mip-NeRF 360 StreetSurf MonoSDF InstantNGP
Closest papers by meaning
- Swift-Mapping: Online Neural Implicit Dense Mapping in Urban Scenes (AAAI 2024 · similarity 0.88)
- Effective Multi-sensor Conditioning for Street-view Novel-view Synthesis (NeurIPS 2026 · similarity 0.87)
- SS3DM: Benchmarking Street-View Surface Reconstruction with a Synthetic 3D Mesh Dataset (NeurIPS 2024 · similarity 0.86)
- LidaRF: Delving into Lidar for Neural Radiance Field on Street Scenes (CVPR 2024 · similarity 0.86)
- GenAssets: Generating in-the-wild 3D Assets in Latent Space (CVPR 2025 · similarity 0.86)
- UniPAD: A Universal Pre-training Paradigm for Autonomous Driving (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.