Research map / Point clouds / Segmentation and detection
Learnable Earth Parser: Discovering 3D Prototypes in Aerial Scans
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Unsupervised probabilistic reconstruction model that decomposes large aerial LiDAR scans into a small set of learned 3D prototypes, enabling instance and low-shot semantic segmentation, plus a new 7-scene Earth Parser Dataset.
Problem
Parsing large, uncurated aerial 3D LiDAR scans into interpretable shape components without annotations, which existing unsupervised decomposition methods (designed for synthetic single objects) do not handle.
Approach
A probabilistic slot-based reconstruction model that selects, affinely deforms, and places learned free-form 3D prototype point clouds (with intensity) to reconstruct a point cloud, trained with asymmetric Chamfer-based accuracy and coverage losses plus regularizers and curriculum learning.
Key results
- Best reconstruction Chamfer distance in 6 of 7 Earth Parser scenes and mIoU of 93.4 on average vs 25.9 for DTI-Sprites and 48.1 for AtlasNet v2 (average column).
- On ShapeNet planes, mIoU 82.7 vs 54.9 for AtlasNet v2 and Chamfer 0.29 vs 0.31.
- Instance counting on Marina boats yields mean relative error of 7.4%, 2.6 times lower than DTI-Sprites.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Learnable Earth Parser instead of the strongest reported baseline | Learnable Earth Parser · Earth Parser Dataset, seven scenes | mIoU: 96.9 vs 93.8; 83.7 vs 71.5; 91.3 vs 39.3; 78.7 vs 41.4; 52.2 vs 42.8; 83.2 vs 56.5; 93.4 vs 87.6 | Helped |
| Use Learnable Earth Parser instead of the strongest reconstruction baseline | Learnable Earth Parser · Earth Parser Dataset, seven scenes | Cham.: 0.72 vs 0.86; 0.88 vs 1.04; 0.40 vs 0.56; 0.82 vs 0.93; 0.44 vs 0.45; 0.29 vs 0.63; 6.65 vs 9.47 | Mixed |
| Remove prototype-selection post-processing (ablation) | Learnable Earth Parser · Urban | Cham.: 0.29 -> 0.28 | Helped |
| Remove prototype-selection post-processing (ablation) | Learnable Earth Parser · Marina | Cham.: 0.82 -> 0.96 | Hurt |
| Add anisotropic scaling (ablation) | Learnable Earth Parser · Urban | Cham.: 0.33 -> 0.29 | Helped |
| Add anisotropic scaling (ablation) | Learnable Earth Parser · Marina | mIoU: 67.2 -> 78.7 | Helped |
| Add learnable prototypes (ablation) | Learnable Earth Parser · Urban | mIoU: 68.3 -> 83.2 | Helped |
| Add learnable prototypes (ablation) | Learnable Earth Parser · Marina | Cham.: 1.07 -> 0.82 | Helped |
| Add learnable prototype scales (ablation) | Learnable Earth Parser · Urban | Cham.: 0.55 -> 0.29 | Helped |
| Add learnable prototype scales (ablation) | Learnable Earth Parser · Marina | mIoU: 40.8 -> 78.7 | Helped |
| Add prototype intensities (ablation) | Learnable Earth Parser · Urban | mIoU: 58.7 -> 83.2 | Helped |
| Add prototype activation regularization Lproto (ablation) | Learnable Earth Parser · Marina | mIoU: 40.7 -> 78.7 | Helped |
Surprising findings
- A simple k-means baseline on intensity and elevation is a surprisingly strong semantic segmentation baseline, beating other baselines in 5 of 7 annotated scenes.
- Reducing the number of prototypes K from 6 to 3 drops mIoU by 26.3 points while barely affecting reconstruction, whereas reducing slots strongly affects reconstruction.
Limitations
- Model is trained separately for each scene rather than generalizing across scenes.
- Semantic segmentation requires manual annotation of prototypes by an operator.
- Reconstruction is not always the most faithful (e.g., Power Plant and Windturbines cases where baselines are better).
Experimental setup
Datasets: Earth Parser Dataset ShapeNet-Part DALES
Benchmarks: Earth Parser Dataset ShapeNet-Part DALES
Metrics: Chamfer distance mIoU mean relative error
Baselines: k-means SuperQuadrics DTI-Sprites AtlasNet v2
Builds on: SuperQuadrics (Paschalidou et al.) Slot Attention DTI-Sprites (Monnier et al.) AtlasNet v2 Loiseau et al. prototype model
Closest papers by meaning
- SynRS3D: A Synthetic Dataset for Global 3D Semantic Understanding from Monocular Remote Sensing Imagery (NeurIPS 2024 · similarity 0.86)
- LiDAR-LLM: Exploring the Potential of Large Language Models for 3D LiDAR Understanding (AAAI 2025 · similarity 0.86)
- TREND: Unsupervised 3D Representation Learning via Temporal Forecasting for LiDAR Perception (NeurIPS 2025 · similarity 0.86)
- USCILab3D: A Large-scale, Long-term, Semantically Annotated Outdoor Dataset (NeurIPS 2024 · similarity 0.86)
- Interpretable3D: An Ad-Hoc Interpretable Classifier for 3D Point Clouds (AAAI 2024 · similarity 0.85)
- Large Spatial Model: End-to-end Unposed Images to Semantic 3D (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.