Research map / 3D vision / 3D generation
Bi-SSC: Geometric-Semantic Bidirectional Fusion for Camera-based 3D Semantic Scene Completion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Bi-SSC, a stereo camera-only semantic scene completion model that fuses geometric and semantic features via Spatial Sensory Fusion and Cross-view Fusion, reaching SOTA on SemanticKITTI.
Problem
Camera-based semantic scene completion struggles to infer geometry and semantics in occluded and shaded regions due to occlusion and lighting ambiguity.
Approach
Combines geometric and semantic image feature branches with a masked, depth-weighted attention module (Spatial Sensory Fusion) within a view and a cross-attention module (Cross-view Fusion) across stereo views, then lifts the result to BEV and voxels with MIA aggregation.
Key results
- Bi-SSC achieves 45.10 IoU and 16.73 mIoU on the SemanticKITTI hidden test set, versus 43.21 IoU and 13.41 mIoU for VoxFormer-T.
- Compared with OccFormer (mono), Bi-SSC improves mIoU from 12.32 to 16.73 and IoU from 34.53 to 45.10.
- On SemanticKITTI-Complex, Bi-SSC drops only 0.39 mIoU versus 0.75 for VoxFormer-T and 1.50 for StereoScene.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use Bi-SSC instead of StereoScene | camera-based SSC · SemanticKITTI hidden test set | IoU: 43.34 -> 45.10 | Helped |
| use Bi-SSC instead of StereoScene | camera-based SSC · SemanticKITTI hidden test set | mIoU: 15.36 -> 16.73 | Helped |
| add Spatial Sensory Fusion (SSF) (ablation) | Bi-SSC · SemanticKITTI val | IoU: 44.38 -> 44.88 | Helped |
| add Spatial Sensory Fusion (SSF) (ablation) | Bi-SSC · SemanticKITTI val | mIoU: 15.35 -> 16.39 | Helped |
| add Cross-view Fusion (CVF) (ablation) | Bi-SSC · SemanticKITTI val | IoU: 44.01 -> 44.88 | Helped |
| add Cross-view Fusion (CVF) (ablation) | Bi-SSC · SemanticKITTI val | mIoU: 15.85 -> 16.39 | Helped |
| add Mutual Interactive Aggregation (MIA) (ablation) | Bi-SSC · SemanticKITTI val | IoU: 44.77 -> 44.88 | Helped |
| add Mutual Interactive Aggregation (MIA) (ablation) | Bi-SSC · SemanticKITTI val | mIoU: 16.21 -> 16.39 | Helped |
| include the attention score in SSF (ablation) | SSF · SemanticKITTI val | mIoU: 15.94 -> 16.39 | Helped |
| use dual query in SSF (ablation) | SSF · SemanticKITTI val | mIoU: 15.75 -> 16.39 | Helped |
| include the mask in SSF (ablation) | SSF · SemanticKITTI val | mIoU: 15.52 -> 16.39 | Helped |
| replace standard Attention with SSF (ablation) | Spatial Sensory Fusion · SemanticKITTI val | IoU: 43.56 -> 44.88 | Helped |
Limitations
- Ablation on the validation set reports different numbers (44.88 IoU / 16.39 mIoU) than the test set results, and the SemanticKITTI-Complex subset is small (300 images) and hand-selected.
- Relies on stereo input (cam2 and cam3), not purely monocular camera setups.
- Paper gives limited detail on the masking and attention-score formulations.
Experimental setup
Datasets: SemanticKITTI KITTI Odometry
Benchmarks: SemanticKITTI hidden test set SemanticKITTI validation set SemanticKITTI-Complex
Metrics: IoU mIoU
Baselines: MonoScene VoxFormer-T OccFormer TPVFormer StereoScene
Builds on: MonoScene StereoScene LSS AgentFormer MIA
Closest papers by meaning
- VLScene: Vision-Language Guidance Distillation for Camera-Based 3D Semantic Scene Completion (AAAI 2025 · similarity 0.89)
- Memory-Augmented Re-Completion for 3D Semantic Scene Completion (AAAI 2025 · similarity 0.88)
- H2GFormer: Horizontal-to-Global Voxel Transformer for 3D Semantic Scene Completion (AAAI 2024 · similarity 0.88)
- Three Cars Approaching within 100m! Enhancing Distant Geometry by Tri-Axis Voxel Scanning for Camera-based Semantic Scene Completion (CVPR 2025 · similarity 0.88)
- IPFormer: Visual 3D Panoptic Scene Completion with Context-Adaptive Instance Proposals (NeurIPS 2025 · similarity 0.88)
- Context and Geometry Aware Voxel Transformer for Semantic Scene Completion (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.