Research map / Architectures and efficiency / CNN design
LSK3DNet: Towards Effective and Efficient 3D Perception with Large Sparse Kernels
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LSK3DNet, a sparse-convolution 3D perception network that scales 3D kernels to 9x9x9 via dynamic sparse training (SDS) and expands width while pruning channels at inference (CWS), reaching SOTA on SemanticKITTI.
Problem
Large 3D kernels improve LiDAR and point cloud perception but cause cubically growing parameter and compute cost; existing large-kernel 3D methods (LargeKernel3D) carry redundant weights and degrade beyond 7x7x7.
Approach
Learn large sparse 3D kernels from scratch with spatial-group dynamic sparse training (prune and regrow within each spatial group), and expand channel width during training then select the most salient channels at inference so model size stays fixed.
Key results
- SemanticKITTI single-scan test mIoU 75.6% (state of the art), with 89 ms/scene inference.
- Compared with naive dense 9x9x9 large kernel, LSK3DNet reduces parameters from 47.9M to 28.8M (about 40%) and FLOPs from 1916.3G to 763.6G (about 60%) while improving mIoU from 67.5 to 70.2 on the validation set.
- ScanNet v2 test mIoU 75.5 (ScanNet20) and 33.1 val on ScanNet200, improving over LargeKernel3D by 1.6% on test.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace the 2DPASS model with LSK3DNet | SemanticKITTI single-scan segmentation · SemanticKITTI single-scan test | mIoU: 74.8 -> 75.6 | Helped |
| replace 2DPASS with LSK3DNet | SemanticKITTI multi-scan segmentation · SemanticKITTI multi-scan test | mIoU: 62.4 -> 63.4 | Helped |
| replace Point Transformer v2 with LSK3DNet | ScanNet segmentation · ScanNet20 val | mIoU: 75.4 -> 75.7 | Helped |
| replace LargeKernel3D with LSK3DNet | KITTI car detection · KITTI val car, Moderate | 3D AP (IoU=0.7): 85.07 -> 85.61 | Helped |
| use 9×9×9 sparse kernels with SDS and CWS instead of dense 9×9×9 kernels (ablation) | Modified SPVCNN · SemanticKITTI val | mIoU: 67.5 -> 70.2 | Helped |
| add SDS to the widened model (ablation) | Modified SPVCNN with 1.8×D width · SemanticKITTI val | mIoU: 67.2 -> 69.3 | Helped |
| add CWS to the widened model (ablation) | Modified SPVCNN with 1.8×D width · SemanticKITTI val | mIoU: 67.2 -> 69.1 | Helped |
| increase kernel size from 5×5×5 to 9×9×9 in LSK3DNet (ablation) | Modified SPVCNN · SemanticKITTI val | mIoU: 66.8 -> 70.2 | Helped |
| increase kernel size from 9×9×9 to 11×11×11 in LSK3DNet (ablation) | Modified SPVCNN · SemanticKITTI val | mIoU: 70.2 -> 70.1 | Hurt |
| set SDS sparsity to 0.4 rather than 0.8 (ablation) | LSK3DNet · SemanticKITTI val | mIoU: 67.2 -> 70.2 | Helped |
| set SDS adaptation frequency fa to 2000 rather than 100 (ablation) | LSK3DNet · SemanticKITTI val | mIoU: 68.8 -> 70.2 | Helped |
| set SDS prune rate p to 0.3 rather than 0.7 (ablation) | LSK3DNet · SemanticKITTI val | mIoU: 69.7 -> 70.2 | Helped |
Surprising findings
- Dense large kernels degrade beyond 9x9x9 and naive dense 1.8x width with larger kernels does not improve performance due to overparameterization and overfitting.
- LargeKernel3D's performance drops as kernel size grows, the opposite trend to the proposed method, which the authors attribute to its U-Net design.
Limitations
- Depth-wise group convolution was found unsuitable for large 3D kernels, restricting the design space.
- Training cost is not reduced: training speed of LSK3DNet is similar to the dense baseline (about 467 ms vs 463 ms per scene).
Experimental setup
Datasets: SemanticKITTI ScanNet v2 KITTI
Benchmarks: SemanticKITTI single-scan SemanticKITTI multi-scan ScanNet v2 ScanNet20 ScanNet200 KITTI car 3D detection
Metrics: mIoU Acc AP (11 recall positions) parameters FLOPs inference speed
Baselines: LargeKernel3D 2DPASS SphereFormer Cylinder3D RPVNet PV-KD MinkUNet Point Transformer v2 Voxel R-CNN SPVNAS
Builds on: LargeKernel3D SLaK Modified SPVCNN Voxel R-CNN Dynamic Sparse Training
Closest papers by meaning
- TREND: Unsupervised 3D Representation Learning via Temporal Forecasting for LiDAR Perception (NeurIPS 2025 · similarity 0.88)
- NBA3D: Neighbor-Based Confidence Adjustment for 3D Rare Object Detection Using LiDAR (AAAI 2025 · similarity 0.87)
- SAFDNet: A Simple and Effective Network for Fully Sparse 3D Object Detection (CVPR 2024 · similarity 0.87)
- Improving Distant 3D Object Detection Using 2D Box Supervision (CVPR 2024 · similarity 0.87)
- An Empirical Study of the Generalization Ability of Lidar 3D Object Detectors to Unseen Domains (CVPR 2024 · similarity 0.87)
- SwiftPillars: High-Efficiency Pillar Encoder for Lidar-Based 3D Detection (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.