Research map / Detection and segmentation / Open-set and few-shot detection
Geometric Exploitation for Indoor Panoramic Semantic Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Splits indoor panoramic semantic segmentation into over-sampled (floor/ceiling, jointly learned with depth) and under-sampled (other objects, decoded by a transformer context module using geometric cues including a new vertical relative distance) segments, achieving SOTA on Stanford2D3DS, Structured3D and Matterport3D.
Problem
Panoramic semantic segmentation of indoor scenes suffers from equirectangular distortion that oversamples floors/ceilings and undersamples small objects, plus limited annotated data and monocular ambiguity.
Approach
Decompose the panorama into over-sampled planar segments (floor, ceiling) estimated jointly with dense depth, and under-sampled segments decoded by a hybrid decoder: a transformer context module fed with global features, distilled depth features, point-cloud features, a novel vertical relative distance (point distances to fitted floor and ceiling planes), and floor-ceiling masks, plus a high-resol
Key results
- Stanford2D3DS fold 1 mIoU 56.8% (vs 53.6% Trans4PASS+ baseline and 56.4% SGAT4PASS)
- Matterport3D test mIoU 33.06% and Structured3D test mIoU 71.66% under RGB-only input, new state of the art among RGB methods
- Ablation: adding vertical relative distance and point cloud features raises fold-1 mIoU from 55.32 to 56.68
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the proposed geometric-exploitation framework instead of SGAT4PASS | Indoor panoramic semantic segmentation · Stanford2D3DS validation (3-fold average) | mIoU: 55.3 -> 55.5 | Helped |
| Use the proposed geometric-exploitation framework instead of SGAT4PASS | Indoor panoramic semantic segmentation · Stanford2D3DS validation (fold 1) | mIoU: 56.4 -> 56.8 | Helped |
| Use the proposed framework instead of the strongest listed RGBD baseline | Indoor panoramic semantic segmentation · Structured3D test | mIoU: 70.17 -> 71.66 | Helped |
| Use the proposed RGB framework instead of the strongest listed RGBD baseline | Indoor panoramic semantic segmentation · Matterport3D validation | mIoU: 39.19 -> 36.42 | Hurt |
| Use the proposed RGB framework instead of the strongest listed RGBD baseline | Indoor panoramic semantic segmentation · Matterport3D test | mIoU: 35.92 -> 33.06 | Hurt |
| Add cross-task feature Fhid to the context module (ablation) | Ours · Stanford2D3DS fold 1 | mIoU: 54.41 -> 54.83 | Helped |
| Add point-cloud representation Fpc to the context module (ablation) | Ours · Stanford2D3DS fold 1 | mIoU: 54.83 -> 55.32 | Helped |
| Add vertical relative distance Fdist to the context module (ablation) | Ours · Stanford2D3DS fold 1 | mIoU: 55.32 -> 56.68 | Helped |
| Add floor-ceiling mask Fm to the context module (ablation) | Ours · Stanford2D3DS fold 1 | mIoU: 56.68 -> 56.80 | Helped |
| Add full depth involvement, including depth estimation, Fpc, and Fdist (ablation) | Ours · Stanford2D3DS (3-fold average) | mIoU: 54.3 -> 55.5 | Helped |
| Add full depth involvement, including depth estimation, Fpc, and Fdist (ablation) | Ours · Stanford2D3DS fold 1 | mIoU: 54.7 -> 56.8 | Helped |
| Add Fpc and Fdist while keeping depth estimation (ablation) | Ours · Stanford2D3DS (3-fold average) | mIoU: 54.6 -> 55.5 | Helped |
Limitations
- Separate estimation of the two segment groups leaves unknown areas after merging
- Depth branch considerably increases model complexity
- Performance on Matterport3D remains limited
Experimental setup
Datasets: Stanford2D3DS Structured3D Matterport3D
Benchmarks: Stanford2D3DS Structured3D Matterport3D
Metrics: mIoU pixel accuracy MRE MAE RMSE delta accuracy
Baselines: Trans4PASS+ SGAT4PASS SFSS-MMSI PanoFormer HoHoNet Trans4PASS FreDSNet
Builds on: Trans4PASS+ PSN plane surface normal loss
Closest papers by meaning
- PanoContext-Former: Panoramic Total Scene Understanding with a Transformer (CVPR 2024 · similarity 0.88)
- AdapToPASS: Ambiguity-aware Adaptive Spherical Transformer for Panoramic Semantic Segmentation (NeurIPS 2026 · similarity 0.87)
- Semantics Distortion and Style Matter: Towards Source-free UDA for Panoramic Segmentation (CVPR 2024 · similarity 0.87)
- GoodSAM: Bridging Domain and Capacity Gaps via Segment Anything Model for Distortion-aware Panoramic Semantic Segmentation (CVPR 2024 · similarity 0.86)
- Unified Panoramic Geometry Estimation via Multi-View Foundation Models (NeurIPS 2026 · similarity 0.86)
- PanoOcc: Unified Occupancy Representation for Camera-based 3D Panoptic Segmentation (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.