Research map / 3D vision / 3D generation
SOAP: Vision-Centric 3D Semantic Scene Completion with Scene-Adaptive Decoder and Occluded Region-Aware View Projection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SOAP, a vision-centric 3D semantic scene completion method with an occluded region-aware view projection that refines occluded voxel features using historical frames, and a scene-adaptive decoder that conditions query tokens on a novelty-based semantic repository.
Problem
Vision-centric 3D semantic scene completion suffers from duplicated voxel features behind occluders during 2D-to-3D view projection, and static query embeddings in the decoder fail to adapt to diverse driving scene appearances.
Approach
Identify occluded voxels from the estimated depth distribution and refine them with deformable cross-attention over historical voxel features; additionally, build a class-wise semantic repository of novelty-selected tokens that scene-adaptive decoder queries attend to.
Key results
- On SemanticKITTI test, SOAP reaches 19.09 SSC mIoU and 46.09 SC IoU, improving over HTCL by +2 mIoU and +1.86 IoU.
- On SSCBench-KITTI360 test, SOAP reaches 20.92 mIoU and 48.12 IoU, surpassing SGN by +2.67 mIoU and +1.06 IoU.
- Ablation on SemanticKITTI val: removing both OAP and SAD drops mIoU from 19.21 to 16.86.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use SOAP instead of HTCL | vision-centric 3D SSC · SemanticKITTI test | SSC mIoU: 17.09 -> 19.09 | Helped |
| use SOAP instead of SGN | vision-centric 3D SSC · SSCBench-KITTI360 test | SSC mIoU: 18.25 -> 20.92 | Helped |
| add OAP (ablation) | SOAP without OAP or SAD · SemanticKITTI validation | SSC mIoU: 16.86 -> 17.22 | Helped |
| add SAD (ablation) | SOAP without OAP or SAD · SemanticKITTI validation | SSC mIoU: 16.86 -> 18.99 | Helped |
| add OAP (ablation) | SOAP with SAD and without OAP · SemanticKITTI validation | SSC mIoU: 18.99 -> 19.21 | Helped |
| add SAD (ablation) | SOAP with OAP and without SAD · SemanticKITTI validation | SSC mIoU: 17.22 -> 19.21 | Helped |
| replace OAP with direct addition of historical voxel features (ablation) | SOAP · SemanticKITTI validation | SSC mIoU: 19.21 -> 18.05 | Hurt |
| replace OAP with standard DCA without occluded-region separation (ablation) | SOAP · SemanticKITTI validation | SSC mIoU: 19.21 -> 18.53 | Hurt |
Surprising findings
- Without the occluded region-aware projection, SOAP still achieves reasonable performance (about 17 mIoU on validation), implying the decoder alone yields a sizable gain without temporal information.
Limitations
- Relies on an off-the-shelf stereo depth model for depth estimation, so occlusion identification depends on depth quality.
- Uses multiple previous frames (P=4), adding temporal dependence and memory cost.
- Evaluated only on two outdoor driving datasets.
Experimental setup
Datasets: SemanticKITTI SSCBench-KITTI360
Benchmarks: SemanticKITTI SSCBench-KITTI360
Metrics: mIoU IoU
Baselines: MonoScene TPVFormer SurroundOcc VoxFormer OccFormer NDC-Scene LowRankOcc Symphonies HASSC SGN HTCL
Builds on: LSS Mask2Former VoxFormer Deformable DETR
Closest papers by meaning
- Memory-Augmented Re-Completion for 3D Semantic Scene Completion (AAAI 2025 · similarity 0.88)
- Three Cars Approaching within 100m! Enhancing Distant Geometry by Tri-Axis Voxel Scanning for Camera-based Semantic Scene Completion (CVPR 2025 · similarity 0.87)
- Not All Voxels Are Equal: Hardness-Aware Semantic Scene Completion with Self-Distillation (CVPR 2024 · similarity 0.86)
- VLScene: Vision-Language Guidance Distillation for Camera-Based 3D Semantic Scene Completion (AAAI 2025 · similarity 0.86)
- Bi-SSC: Geometric-Semantic Bidirectional Fusion for Camera-based 3D Semantic Scene Completion (CVPR 2024 · similarity 0.86)
- Context and Geometry Aware Voxel Transformer for Semantic Scene Completion (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.