Research map / 3D vision / 3D generation
SGFormer: Satellite-Ground Fusion for 3D Semantic Scene Completion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SGFormer, a dual-branch satellite-ground fusion framework that uses satellite images alongside ground camera views to improve camera-based 3D semantic scene completion on SemanticKITTI and SSCBench-KITTI-360.
Problem
Camera-based 3D semantic scene completion struggles with visual occlusions and lacks long-range global context for occluded regions.
Approach
A tightly coupled satellite-ground dual-branch architecture where ground-view voxel features warm up satellite BEV queries via deformable self-attention (satellite correction), followed by an adaptive channel- and spatial-weighted fusion of voxel and BEV features.
Key results
- On SemanticKITTI validation, SGFormer reaches 45.01 IoU and 16.68 mIoU, outperforming VoxFormer (44.05 IoU, 12.30 mIoU) and Symphonies (41.92 IoU, 14.89 mIoU).
- On SSCBench-KITTI-360 test, SGFormer reports 18.30 mIoU versus 17.82 for Symphonies among camera-based methods (text cites 46.55 IoU while Table 2 lists 46.35).
- Ablation shows satellite correction raises global-category mIoU from 26.54 to 28.43 and the full model reaches 29.31 global mIoU.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the satellite branch (ablation) | Ground-branch SSC model · SemanticKITTI val | IoU: 44.53 -> 44.00 | Hurt |
| add the satellite branch (ablation) | Ground-branch SSC model · SemanticKITTI val | mIoU: 14.80 -> 15.01 | Helped |
| add satellite correction (ablation) | SGFormer with satellite branch, without fusion · SemanticKITTI val, global category | mIoU: 26.54 -> 28.43 | Helped |
| add adaptive fusion (ablation) | SGFormer with satellite branch, without satellite correction · SemanticKITTI val, detail category | mIoU: 8.26 -> 9.01 | Helped |
| add satellite correction and adaptive fusion (ablation) | SGFormer · SemanticKITTI val | IoU: 44.53 -> 45.01 | Helped |
| add ±5m localization noise (ablation) | SGFormer without satellite correction · SemanticKITTI val | mIoU: 15.81 -> 15.10 | Hurt |
| add ±5m localization noise (ablation) | SGFormer with satellite correction · SemanticKITTI val | mIoU: 16.68 -> 15.96 | Hurt |
| use satellite correction under ±5m localization noise (ablation) | SGFormer · SemanticKITTI val, global category | mIoU: 26.08 -> 28.75 | Helped |
| use satellite correction under ±5m localization noise (ablation) | SGFormer · SemanticKITTI val, detail category | mIoU: 8.70 -> 8.53 | Hurt |
Surprising findings
- Adding the satellite branch alone slightly lowers IoU (44.53 to 44.00) on SemanticKITTI, indicating satellite cues can hurt fine-detail and geometry prediction without correction.
- A 5-meter localization noise reduces mIoU by about 4.3% with satellite correction and 4.5% without, and without correction the global mIoU falls below the ground-only baseline.
Limitations
- Sensitive to satellite localization noise (about 4-5% mIoU drop under ±5 m noise).
- Satellite images are pre-captured and miss fine details, traffic signs, and temporary objects, hurting small/dynamic object prediction.
- Requires GPS-aligned satellite imagery; SemanticKITTI sequence 03 is excluded due to lack of GPS information.
Experimental setup
Datasets: SemanticKITTI SSCBench-KITTI-360 Google Maps satellite images satellite images provided by [30]
Benchmarks: SemanticKITTI SSCBench-KITTI-360
Metrics: IoU mIoU
Baselines: MonoScene TPVFormer VoxFormer OccFormer SurroundOcc Symphonies GaussianFormer LMSCNet JS3C-Net SSCNet
Builds on: VoxFormer BEVFormer deformable attention MonoScene Symphonies
Closest papers by meaning
- IPFormer: Visual 3D Panoptic Scene Completion with Context-Adaptive Instance Proposals (NeurIPS 2025 · similarity 0.88)
- Bi-SSC: Geometric-Semantic Bidirectional Fusion for Camera-based 3D Semantic Scene Completion (CVPR 2024 · similarity 0.87)
- H2GFormer: Horizontal-to-Global Voxel Transformer for 3D Semantic Scene Completion (AAAI 2024 · similarity 0.85)
- Memory-Augmented Re-Completion for 3D Semantic Scene Completion (AAAI 2025 · similarity 0.85)
- Satellite to GroundScape - Large-scale Consistent Ground View Generation from Satellite Views (CVPR 2025 · similarity 0.85)
- From Objects to Anywhere: A Holistic Benchmark for Multi-level Visual Grounding in 3D Scenes (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.