Research map / Detection and segmentation / CNN features

AdapToPASS: Ambiguity-aware Adaptive Spherical Transformer for Panoramic Semantic Segmentation

NeurIPS 2026 · Detection and segmentation › CNN features · approach: Plain and high-resolution ViTs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes AdapToPASS, a bio-inspired spherical Transformer for panoramic semantic segmentation with ambiguity-aware attention, a bifocal acuity/lateral representation, and SDF boundary supervision, evaluated for robustness under unseen spherical transformations on Stanford2D3D and WildPASS.

Problem

Spherical Transformers for panoramic semantic segmentation rely on canonical spherical layouts and stable viewpoints, so they degrade under unconstrained camera motion and unseen spherical transformations (rotation, scale, translation, orientation and viewpoint shifts).

Approach

Adaptive Spherical Attention predicts a per-token contextual ambiguity score that rescales the geodesic distance in the attention bias, combined with a two-stream bifocal representation (full-resolution acuity stream plus downsampled lateral stream) and SDF-based boundary supervision on the icosphere.

Key results

• AdapToPASS reaches 70.80% mIoU on Stanford2D3D and 69.58% on WildPASS, versus 68.79% (SO3UFormer) and 65.40% (Elite360D) for the next-best baselines.
• Under unseen spherical transformations, AdapToPASS reports up to +13.38% relative mIoU gain on Stanford2D3D and +18.77% on WildPASS over the next-best method.
• AdapToPASS-Swift has under 2M parameters and surpasses compact baselines while retaining robustness.

Surprising findings

• Several baselines (e.g., SGAT4PASS, HEAL-SWIN) lose substantial mIoU on WildPASS relative to Stanford2D3D, indicating dataset-dependent robustness.

Limitations

• Performance still degrades notably under the stress setting, especially orientation and viewpoint shifts.
• WildPASS evaluated on only 3 foreground classes (plus background), reducing label space.
• All baselines trained from scratch with RGB only; no transformation-specific augmentation used.

Experimental setup

Datasets: Stanford2D3D WildPASS

Benchmarks: Stanford2D3D WildPASS

Metrics: mIoU mAcc confidence

Baselines: SFSS SGAT4PASS Elite360D HEAL-SWIN SphereUFormer SO3UFormer Trans4PASS HexRUNet SphereUFormer-2M

Builds on: SphereUFormer Trans4PASS SO3UFormer Lovasz-Softmax loss class-balanced loss

Closest papers by meaning

• Geometric Exploitation for Indoor Panoramic Semantic Segmentation (NeurIPS 2024 · similarity 0.87)
• PanoWorld: Towards Spatial Supersensing in 360◦ Panorama World (NeurIPS 2026 · similarity 0.86)
• Semantics Distortion and Style Matter: Towards Source-free UDA for Panoramic Segmentation (CVPR 2024 · similarity 0.86)
• SGFormer: Semantic-Geometry Fusion Transformer for Multi-modal 3D Panoptic Segmentation (AAAI 2025 · similarity 0.86)
• SphereDiffusion: Spherical Geometry-Aware Distortion Resilient Diffusion Model (AAAI 2024 · similarity 0.86)
• Large Spatial Model: End-to-end Unposed Images to Semantic 3D (NeurIPS 2024 · similarity 0.85)