Research map / Point clouds / Segmentation and detection
OA-CNNs: Omni-Adaptive Sparse CNNs for 3D Semantic Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes OA-CNNs, pure sparse CNNs with adaptive receptive fields (pyramid grid partition plus adaptive aggregator) and Adaptive Relation Convolution (ARConv) that match or beat point transformers on 3D semantic segmentation at lower latency.
Problem
Sparse CNNs for 3D semantic segmentation lag behind point transformers in accuracy because they lack adaptivity in receptive field size and relation modeling among local voxels.
Approach
Partition the voxel scene into pyramid grids of multiple sizes, learn per-voxel softmax weights to fuse them (adaptive receptive fields), and generate depthwise kernel weights from each voxel's correlation with its grid centroid (multi-one-multi adaptive relation) to get attention-like adaptivity with linear cost.
Key results
- Reaches 76.1% val mIoU on ScanNet v2 (vs. PTv2 75.4%, Stratified Transformer 74.3%) with no self-attention modules.
- Achieves 78.9% mIoU on nuScenes and 70.6% on SemanticKITTI validation, above SphereFormer (78.4 / 67.8).
- Reports 33.3% mIoU on ScanNet200 validation-free 'All' test column and 71.1% on S3DIS area 5 (below PTv2's 71.6).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use OA-CNNs instead of OctFormer | ScanNet v2 · ScanNet v2 val | mIoU: 75.7 -> 76.1 | Helped |
| Use OA-CNNs instead of OctFormer | ScanNet v2 · ScanNet v2 test | mIoU: 76.6 -> 75.6 | Hurt |
| Use OA-CNNs instead of SphereFormer | 3D semantic segmentation · nuScenes | mIoU: 78.4 -> 78.9 | Helped |
| Use OA-CNNs instead of SphereFormer | 3D semantic segmentation · SemanticKITTI | mIoU: 67.8 -> 70.6 | Helped |
| Use OA-CNNs instead of PTv2 | 3D semantic segmentation · ScanNet200 val all | mIoU: 29.3 -> 32.3 | Helped |
| Use OA-CNNs instead of OctFormer | 3D semantic segmentation · ScanNet200 test all | mIoU: 32.6 -> 33.3 | Helped |
| Use OA-CNNs instead of PTv2 | 3D semantic segmentation · S3DIS area 5 | mIoU: 71.6 -> 71.1 | Hurt |
| Add adaptive aggregation rather than single-scale processing (ablation) | OA-CNNs · ScanNet v2 val | mIoU: 75.0 -> 76.1 | Helped |
| Replace concatenation with adaptive aggregation (ablation) | OA-CNNs · ScanNet v2 val | mIoU: 75.2 -> 76.1 | Helped |
| Use 3 rather than 2 pyramid stages (ablation) | OA-CNNs with adaptive aggregation · ScanNet v2 val | mIoU: 75.2 -> 76.1 | Helped |
| Use ARConv instead of multi-head self-attention (ablation) | OA-CNNs · ScanNet v2 val | mIoU: 73.5 -> 76.1 | Helped |
| Use depthwise rather than grouped convolution (ablation) | OA-CNNs · ScanNet v2 val | mIoU: 75.4 -> 76.1 | Helped |
Surprising findings
- On S3DIS area 5, OA-CNNs (71.1 mIoU) trails PTv2 (71.6) and Point Transformer (70.4 is below it, but PTv2 is higher), so the advantage is not uniform across benchmarks.
- Naive regular (dense) dynamic convolution with the generated kernels was unstable and non-convergent, forcing use of depthwise convolution.
Limitations
- Pyramid grid sizes are set empirically rather than searched.
- Dynamic regular convolution was unstable, so depthwise variant is used, limiting kernel expressiveness.
- Does not outperform all point-based methods on every benchmark (e.g., S3DIS).
Experimental setup
Datasets: ScanNet v2 ScanNet200 S3DIS nuScenes SemanticKITTI
Benchmarks: ScanNet v2 ScanNet200 S3DIS area 5 nuScenes SemanticKITTI
Metrics: mIoU latency (ms) GPU memory (G)
Baselines: Point Transformer PTv2 Stratified Transformer OctFormer SparseUNet MinkowskiNet LargeKernel3D SphereFormer SPVNAS Cylinder3D
Builds on: Sparse convolution (MinkowskiNet / SparseUNet) Point Transformer V2 Conditional convolution Pointcept
Closest papers by meaning
- 3D Semantic Segmentation With Submanifold Sparse Convolutional Networks (CVPR 2018 · similarity 0.90)
- Omni3D: A Large Benchmark and Model for 3D Object Detection in the Wild (CVPR 2023 · similarity 0.89)
- ODIN: A Single Model for 2D and 3D Segmentation (CVPR 2024 · similarity 0.87)
- Segment Any 3D Object with Language (ICLR 2025 · similarity 0.87)
- Test-Time Adaptation of Vision-Language Models for Open-Vocabulary Semantic Segmentation (NeurIPS 2025 · similarity 0.86)
- Emergent Open-Vocabulary Semantic Segmentation from Off-the-shelf Vision-Language Models (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.