Research map / 3D vision / 3D generation
SurroundSDF: Implicit 3D Scene Understanding Based on Signed Distance Field
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SurroundSDF, a camera-only query-based implicit model that predicts a signed distance field and semantic field from surround images, trained with a weakly supervised "Sandwich Eikonal" loss using LiDAR points and occupancy GT.
Problem
Vision-centric 3D semantic occupancy prediction from surround cameras uses discrete voxels and cannot produce continuous, accurate obstacle surfaces, and precise SDF ground truth is unavailable for supervision.
Approach
Supervise an SDF field from both sides of the surface: LiDAR points are constrained to SDF=0, occupied voxel grids are pushed to have a negative minimum SDF, and free voxel grids are pushed to positive SDF, combined with the Eikonal constraint; a joint Dice loss over SDF-derived free logits and semantic logits reconciles geometry and semantics.
Key results
- Small version (ResNet50, no TTA) reaches 42.37 mIoU on Occ3D-nuScenes val, above FB-OCC* (42.06) with the same backbone.
- Large version (ConvNeXt-B) reaches 53.01 mIoU, surpassing UniOCC (51.5) and FB-OCC* (52.79, with TTA and InternImage-H).
- Scene reconstruction Abs Rel of 0.174 versus 0.224 for SurroundDepth and 0.259 for R3D3.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SDF scene representation with SIREN supervision instead of occupancy prediction (ablation) | SurroundSDF · Occ3D-nuScenes | Abs-Rel: 0.342 -> 0.188 | Helped |
| Use SDF scene representation with SIREN supervision instead of occupancy prediction (ablation) | SurroundSDF · Occ3D-nuScenes | mIoU: 38.33 -> 31.51 | Hurt |
| Use LODE supervision for SDF instead of occupancy prediction (ablation) | SurroundSDF · Occ3D-nuScenes | Abs-Rel: 0.342 -> 0.192 | Helped |
| Use LODE supervision for SDF instead of occupancy prediction (ablation) | SurroundSDF · Occ3D-nuScenes | mIoU: 38.33 -> 36.66 | Hurt |
| Use Sandwich Eikonal SDF supervision (ablation) | SurroundSDF · Occ3D-nuScenes | mIoU: 38.33 -> 40.52 | Helped |
| Use Sandwich Eikonal SDF supervision (ablation) | SurroundSDF · Occ3D-nuScenes | Abs-Rel: 0.342 -> 0.171 | Helped |
| Add joint supervision combining geometry and semantics (ablation) | SurroundSDF with Sandwich Eikonal · Occ3D-nuScenes | mIoU: 40.52 -> 42.37 | Helped |
| Add joint supervision combining geometry and semantics (ablation) | SurroundSDF with Sandwich Eikonal · Occ3D-nuScenes | Abs-Rel: 0.171 -> 0.174 | Hurt |
| Increase OCC GT voxel size from 0.4m to 0.8m or 1.6m (ablation) | SurroundSDF · Occ3D-nuScenes | mIoU: 42.37 -> 40.74; 42.37 -> 40.16 | Hurt |
| Increase OCC GT voxel size from 0.4m to 0.8m or 1.6m (ablation) | SurroundSDF · 3D scene reconstruction | Abs-Rel: 0.174 -> 0.176; 0.174 -> 0.181 | Hurt |
| Reduce LiDAR point sampling rate to 0.5 (ablation) | SurroundSDF · Occ3D-nuScenes | mIoU: 42.37 -> 42.37 | No clear effect |
| Reduce LiDAR point sampling rate to 0.2 or 0.1 (ablation) | SurroundSDF · Occ3D-nuScenes | mIoU: 42.37 -> 42.16; 42.37 -> 41.07 | Hurt |
Surprising findings
- Greater geometric (occupancy IoU) accuracy does not necessarily yield better semantic mIoU; the optimal SDF thresholds for the two metrics differ significantly.
Limitations
- Performance degrades when the occupancy GT resolution is coarsened or LiDAR density is reduced, though the degradation is smaller than for FB-OCC.
- Relies on occupancy ground truth and LiDAR points for supervision, so it is not fully camera-only in training.
Experimental setup
Datasets: Occ3D-nuScenes nuScenes
Benchmarks: Occ3D-nuScenes
Metrics: mIoU Abs Rel Sq Rel RMSE delta < 1.25
Baselines: SurroundOcc BEVStereo UniOCC MiLO FB-OCC BEVFormer VoxFormer BEVDet SurroundDepth R3D3 SIREN LODE
Builds on: Eikonal formulation SIREN LODE LSS BEVStereo
Closest papers by meaning
- Sensing Surface Patches in Volume Rendering for Inferring Signed Distance Functions (AAAI 2025 · similarity 0.85)
- Not All Voxels Are Equal: Hardness-Aware Semantic Scene Completion with Self-Distillation (CVPR 2024 · similarity 0.85)
- S2MDF: A Plug-And-Play Layer for Intersection-Free Multi-Object Signed Distance Fields (NeurIPS 2026 · similarity 0.85)
- Achieving Speed-Accuracy Balance in Vision-based 3D Occupancy Prediction via Geometric-Semantic Disentanglement (AAAI 2025 · similarity 0.85)
- DUDF: Differentiable Unsigned Distance Fields with Hyperbolic Scaling (CVPR 2024 · similarity 0.84)
- Semi-supervised 3D Semantic Scene Completion with 2D Vision Foundation Model Guidance (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.