Research map / 3D vision / 3D generation
HUSH: Holistic Panoramic 3D Scene Understanding using Spherical Harmonics
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HUSH, a panoramic indoor 3D scene understanding framework that uses scene-adapted spherical harmonic bases as attention queries plus an SH basis index module for depth, normal, and layout estimation.
Problem
Panoramic (360°) 3D scene understanding such as depth and surface normal estimation suffers from ERP distortion, and patch-based projection methods are computationally expensive.
Approach
Treat spherical harmonic bases as implicit structural priors: scale them per scene with a predicted coefficient extractor, use them as cross-attention queries over multi-scale image features, and select task-relevant bases via a cosine-similarity index map.
Key results
- On Matterport3D, Ours (D+N) reaches Abs Rel 0.0838 and RMSE 0.4164 versus Elite360D's 0.1115 and 0.4875, per the paper's reported 24.1% and 14.52% relative gains.
- On Structured3D, Ours (D+N) achieves Abs Rel 0.0459 and delta1 0.9776, compared with 0.1480 and 0.8741 for Elite360D.
- On Stanford2D3D, the SH-query ablation shows SH queries outperform learnable queries (Abs Rel 0.078 vs 0.080 for the learnable-query variant).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SH bases rather than learnable queries in the hierarchical attention module (ablation) | HUSH · Stanford2D3D | Abs Rel | Helped |
| Add the SH basis index module (ablation) | HUSH · Stanford2D3D | RMSE: 0.336 -> 0.333 | Helped |
| Add the SH basis index module (ablation) | HUSH · Stanford2D3D | Abs Rel: 0.081 -> 0.078 | Helped |
| Add the SH coefficient extractor for scene-adaptive SH bases (ablation) | HUSH · Stanford2D3D | Abs Rel: 0.080 -> 0.078 | Helped |
| Use SH level 5 (15 bases) instead of SH level 10 (55 bases) (ablation) | HUSH · Stanford2D3D | RMSE: 0.333 -> 0.337 | No clear effect |
| Train with depth and surface normal jointly instead of depth only | HUSH · Stanford2D3D | RMSE: 0.3418 -> 0.3332 | Helped |
Surprising findings
- A variant with only 5 SH levels (15 bases) performs nearly as well as the 10-level (55 bases) full model, suggesting robustness to the number of bases.
Limitations
- Layout estimation is only shown as a qualitative extension with comparison to layout-only methods, without quantitative tables in the main text.
- Evaluated only on indoor panorama datasets with depth/normal ground truth.
Experimental setup
Datasets: Matterport3D Stanford2D3D Structured3D MatterportLayout
Benchmarks: Matterport3D Stanford2D3D Structured3D MatterportLayout
Metrics: Abs Rel Sq Rel RMSE delta1 delta2 delta3
Baselines: UniFuse PanoFormer HRDFuse EGformer Elite360D Elite360M
Builds on: Plane2Depth iDisc Swin Transformer Multi-scale deformable attention BerHu loss
Closest papers by meaning
- PanoContext-Former: Panoramic Total Scene Understanding with a Transformer (CVPR 2024 · similarity 0.87)
- Splatter-360: Generalizable 360 Gaussian Splatting for Wide-baseline Panoramic Images (CVPR 2025 · similarity 0.85)
- AdapToPASS: Ambiguity-aware Adaptive Spherical Transformer for Panoramic Semantic Segmentation (NeurIPS 2026 · similarity 0.85)
- PanoWorld: Towards Spatial Supersensing in 360◦ Panorama World (NeurIPS 2026 · similarity 0.85)
- Unified Panoramic Geometry Estimation via Multi-View Foundation Models (NeurIPS 2026 · similarity 0.84)
- OneCanvas: 3D Scene Understanding via Panoramic Reprojection (NeurIPS 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.