Research map / 3D vision / Depth and stereo
3D Building Reconstruction from Monocular Remote Sensing Images with Multi-level Supervisions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MLS-BRN, a monocular remote-sensing building reconstruction network that trains jointly on samples with footprint-only, footprint+height, and footprint+offset+height annotations using pseudo bounding boxes and roof-offset guided footprint extraction.
Problem
Monocular 3D building reconstruction from off-nadir remote sensing images normally needs expensive fully-annotated 3D labels (footprint, roof-to-footprint offset, height), which limits cross-city scale-up.
Approach
Extends LOFT-FOA with a Pseudo Building Bbox Calculator (predicts off-nadir and offset angles to build pseudo bboxes for samples lacking bbox/offset labels) and a Roof-Offset guided Footprint Extractor (predicts footprint masks from predicted roof and offset), plus a three-level training strategy that mixes samples with different annotation levels.
Key results
- On BN100, MLS-BRN reaches 66.36 F1 and 4.76 EPE versus LOFT-FOA's 64.31 F1 and 4.94 EPE.
- Trained on BN30/70 (30% offset-labeled), MLS-BRN reaches 65.49 F1 versus LOFT-FOA's 61.35 F1 trained on BN30.
- On OmniCity-view3 height estimation, MLS-BRN achieves height MAE 10.90 and RMSE 21.09, versus LOFT-FOA+H at 11.12 and 21.60.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MLS-BRN instead of LOFT-FOA for footprint extraction | BONAI · BONAI test (BN100) | F1-score: 64.31 -> 66.36 | Helped |
| Use MLS-BRN instead of LOFT-FOA for footprint extraction | OmniCity-view3 · OmniCity-view3 test (OC100) | F1-score: 70.46 -> 72.25 | Helped |
| Train with multi-level samples (X OH+X H) using MLS-BRN rather than only X OH with LOFT-FOA | BONAI · BN30/70 versus BN30 | F1-score: 61.35 -> 65.49 | Helped |
| Train with multi-level samples (X OH+X H) using MLS-BRN rather than only X OH with LOFT-FOA | OmniCity-view3 · OC30/70 versus OC30 | F1-score: 67.09 -> 70.53 | Helped |
| Train with multi-level samples (X OH+X H+X N) using MLS-BRN rather than only X OH with LOFT-FOA | BONAI + HK · BH30/70 versus BH30 | F1-score: 54.96 -> 58.57 | Helped |
| Add Pseudo Building Bbox Calculator (PBC) (ablation) | LOFT-FOA on BONAI multi-level setting · BN30/70 | F1-score: 61.35 -> 62.32 | Helped |
| Add Roof-Offset guided Footprint Extractor (ROFE) (ablation) | LOFT-FOA on BONAI multi-level setting · BN30/70 | F1-score: 61.35 -> 62.87 | Helped |
| Add PBC and ROFE together (ablation) | LOFT-FOA on BONAI multi-level setting · BN30/70 | F1-score: 61.35 -> 65.40 | Helped |
| Add PBC and ROFE together (ablation) | LOFT-FOA on BONAI multi-level setting · BN30/70 | EPE: 5.70 -> 5.49 | Helped |
| Use MLS-BRN instead of LOFT-FOA for height prediction | OmniCity-view3 · OmniCity-view3 test | height MAE: 11.12 -> 10.90 | Helped |
| Use MLS-BRN instead of LOFT-FOA for height prediction | OmniCity-view3 · OmniCity-view3 test | height RMSE: 21.60 -> 21.09 | Helped |
| Add X N samples to X OH+X H training (ablation) | MLS-BRN on OmniCity-view3 · OmniCity-view3 test | F1-score: 70.37 -> 70.81 | Helped |
Limitations
- Height estimation results are compared mainly against LOFT-FOA+H, a baseline the authors constructed themselves.
- Final 3D models rely on an external footprint regularization method [38] applied in post-processing.
- Gains on the BH100 setting are marginal (60.92 vs 60.85 F1).
Experimental setup
Datasets: BONAI OmniCity-view3 HK
Benchmarks: BONAI OmniCity-view3 HK
Metrics: F1-score precision recall EPE height MAE height RMSE
Baselines: LOFT-FOA MTBR-Net PANet HRNetv2 Mask R-CNN Cascade Mask R-CNN SARPN DORN
Builds on: LOFT-FOA Mask R-CNN Feature Pyramid Network
Closest papers by meaning
- SynRS3D: A Synthetic Dataset for Global 3D Semantic Understanding from Monocular Remote Sensing Imagery (NeurIPS 2024 · similarity 0.83)
- USCILab3D: A Large-scale, Long-term, Semantically Annotated Outdoor Dataset (NeurIPS 2024 · similarity 0.81)
- Learnable Earth Parser: Discovering 3D Prototypes in Aerial Scans (CVPR 2024 · similarity 0.81)
- Aerial Lifting: Neural Urban Semantic and Building Instance Lifting from Aerial Imagery (CVPR 2024 · similarity 0.81)
- PBWR: Parametric-Building-Wireframe Reconstruction from Aerial LiDAR Point Clouds (CVPR 2024 · similarity 0.81)
- ArcPro: Architectural Programs for Structured 3D Abstraction of Sparse Points (CVPR 2025 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.