Research map / Detection and segmentation / Open-set and few-shot detection
SG-BEV: Satellite-Guided BEV Fusion for Cross-View Semantic Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SG-BEV, a satellite-guided BEV fusion model that maps street-view panorama features into satellite BEV space and fuses them with satellite features for fine-grained building attribute (land use, floor count) segmentation.
Problem
Fine-grained building attribute segmentation (land use, floor number) from paired satellite and street-view images, where existing cross-view projections miss facade features and BEV features concentrate unevenly on building walls.
Approach
Applies BEV lifting to panoramic street-view features and adds a Satellite-Guided Reprojection module that uses satellite building footprints and depth to offset BEV points into building interiors, followed by flow-based alignment and fusion with satellite features.
Key results
- SG-BEV reaches 37.54 mIoU on OmniCity land use and 40.64 on Brooklyn floor, versus 32.17 and 30.81 for BEVFormer (cross-view baseline).
- Averaged over datasets, mIoU improves by 10.13% over the best satellite-based method and 5.21% over the best cross-view method (as stated in the abstract).
- Adding SGR to SegNext raises mIoU by about 2.40 points over depth-only DGR reprojection on OmniCity floor/land-use ablation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SG-BEV instead of the strongest satellite-based segmentation method | SG-BEV · OmniCity, Brooklyn, Boston and Vigor | mIOU: +10.13% | Helped |
| Use SG-BEV instead of the strongest cross-view method | SG-BEV · OmniCity, Brooklyn, Boston and Vigor | mIOU: +5.21% | Helped |
| Add depth-only Depth-Guided Reprojection (DGR) to direct BEV projection (ablation) | UNet and SegNext · OmniCity and Brooklyn, land use and floor | mIOU: 27.07 -> 32.52; 34.13 -> 37.01; 37.04 -> 42.33; 39.61 -> 45.06; 32.10 -> 35.49; 30.56 -> 37.64; 42.43 -> 45.16; 43.55 -> | Helped |
| Add satellite guidance to DGR, yielding Satellite-Guided Reprojection (SGR) (ablation) | UNet and SegNext · OmniCity and Brooklyn, land use and floor | mIOU: 32.52 -> 36.67; 37.01 -> 40.53; 42.33 -> 47.10; 45.06 -> 48.55; 35.49 -> 37.54; 37.64 -> 40.64; 45.16 -> 47.19; 48.17 -> | Helped |
| Use Cross-View Fusion instead of feature concatenation (ablation) | Cross-view feature fusion module · OmniCity and Brooklyn, land use and floor, without reprojection, with DGR, and with SGR | mIOU: 31.78 -> 32.10; 29.34 -> 30.56; 42.12 -> 42.43; 42.28 -> 43.55; 33.85 -> 35.49; 35.13 -> 37.64; 43.08 -> 45.16; 46.19 -> | Helped |
Surprising findings
- Cross-view fusion module gains are largest with depth-only DGR reprojection and shrink with SGR, because satellite guidance already reduces misalignment.
Limitations
- Relies on monocular depth estimation whose errors affect BEV placement.
- Single-timestamp data lacks temporal information, so BEVFormer's Z-direction expansion is not fully used.
Experimental setup
Datasets: OmniCity Vigor Brooklyn Boston
Benchmarks: OmniCity Vigor Brooklyn Boston
Metrics: mIoU overall accuracy
Baselines: FarSeg SAN SegNext Spherical Transform Geometric Projection BEVFormer
Builds on: BEVFormer Lift, Splat, Shoot SegNeXt
Closest papers by meaning
- PC-BEV: An Efficient Polar-Cartesian BEV Fusion Framework for LiDAR Semantic Segmentation (AAAI 2025 · similarity 0.85)
- Aerial Lifting: Neural Urban Semantic and Building Instance Lifting from Aerial Imagery (CVPR 2024 · similarity 0.84)
- SatSynth: Augmenting Image-Mask Pairs through Diffusion Models for Aerial Semantic Segmentation (CVPR 2024 · similarity 0.83)
- VQ-Map: Bird's-Eye-View Map Layout Estimation in Tokenized Discrete Space via Vector Quantization (NeurIPS 2024 · similarity 0.83)
- Improving Bird's Eye View Semantic Segmentation by Task Decomposition (CVPR 2024 · similarity 0.83)
- SGFormer: Satellite-Ground Fusion for 3D Semantic Scene Completion (CVPR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.