Research map / Detection and segmentation / 3D detection and driving
FSHNet: Fully Sparse Hybrid Network for 3D Object Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FSHNet, a fully sparse 3D LiDAR detector that adds slot-partitioned linear attention (SlotFormer), dynamic sparse label assignment, and sparse upsampling to sparse convolution encoders, reaching SOTA on Waymo, nuScenes, and Argoverse2.
Problem
Fully sparse 3D object detectors only compute features on non-empty voxels, which breaks long-range interactions between distant voxels and causes missing center features that hinder optimization.
Approach
Partition sparse voxels into slots spanning the full scene along one axis, apply linear attention within each slot with alternating axes for global interactions, assign positive samples dynamically from the nearest candidate voxels to each object center using a cost and IoU-based top-k rule, and recover fine detail with a sparse upsampling layer.
Key results
- On the Waymo Open validation set, FSHNetbase reaches 77.1/74.9 vehicle/pedestrian-level L2 mAP/mAPH overall (per Table 1), surpassing SAFDNet (75.7/73.9).
- On nuScenes validation, FSHNetbase achieves 71.7 NDS and 68.1 mAP, above SAFDNet's 71.0 NDS and 66.3 mAP.
- On Argoverse2 validation, FSHNetbase reaches 40.2 mAP versus 38.7 for re-implemented SAFDNet.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FSHNetbase instead of SAFDNet | Waymo Open detection · Waymo Open validation | LEVEL 2 mAP/mAPH: 75.7/73.9 -> 77.1/74.9 | Helped |
| Use FSHNetbase instead of DSVT | nuScenes detection · nuScenes validation | NDS: 71.1 -> 71.7 | Helped |
| Use FSHNetbase instead of SAFDNet | Argoverse2 detection · Argoverse2 validation | mAP: 38.7 -> 40.2 | Helped |
| Add SlotFormer (ablation) | VoxelNeXt baseline · Waymo Open validation, 20% training samples | Vehicle AP/APH: 69.1/68.7 -> 70.3/69.9 | Helped |
| Add dynamic sparse label assignment (ablation) | VoxelNeXt baseline · Waymo Open validation, 20% training samples | Vehicle AP/APH: 69.1/68.7 -> 69.9/69.5 | Helped |
| Add sparse upsampling (ablation) | VoxelNeXt baseline · Waymo Open validation, 20% training samples | Pedestrian AP/APH: 75.3/69.5 -> 76.6/71.0 | Helped |
| Use all three components: SlotFormer, dynamic sparse label assignment, and sparse upsampling (ablation) | VoxelNeXt baseline · Waymo Open validation, 20% training samples | Vehicle AP/APH: 69.1/68.7 -> 72.5/72.0 | Helped |
| Use SlotFormer slot partition with linear attention instead of window-and-set partition with self-attention (ablation) | FSHNetlight · Waymo Open validation, 20% training samples | Vehicle AP/APH: 71.8/71.3 -> 72.5/72.0 | Helped |
| Replace no upsampling with SM-SU (ablation) | FSHNetlight · Waymo Open validation, 20% training samples | Pedestrian AP/APH: 75.9/70.4 -> 77.9/72.4 | Helped |
| Replace no upsampling with SP-SU (ablation) | FSHNetlight · Waymo Open validation, 20% training samples | Vehicle AP/APH: 71.4/70.9 -> 72.5/72.0 | Helped |
| Increase DSLA candidate count from 1 to 5 (ablation) | FSHNetlight · Waymo Open validation, 20% training samples | Vehicle AP/APH: 70.8/70.3 -> 72.5/72.0 | Helped |
| Increase DSLA candidate count from 5 to 7 (ablation) | FSHNetlight · Waymo Open validation, 20% training samples | Vehicle AP/APH: 72.5/72.0 -> 72.2/71.7 | Hurt |
Surprising findings
- SlotFormer adds notable latency (about 16 ms for FSHNetlight and 29 ms for FSHNetbase on an RTX 3090), a cost the authors acknowledge.
Limitations
- SlotFormer adds inference latency to the sparse detector.
- Only LiDAR point clouds are used; the method is evaluated only on three driving datasets.
- Dynamic label assignment gains plateau beyond 5 candidate voxels, so the candidate count is a tuned hyperparameter.
Experimental setup
Datasets: Waymo Open nuScenes Argoverse2
Benchmarks: Waymo Open nuScenes Argoverse2
Metrics: mAP mAPH NDS AP
Baselines: VoxelNeXt SAFDNet FSDv1 FSDv2 DSVT ScatterFormer HEDNet CenterPoint LION-Mamba DCDet
Builds on: VoxelNeXt SAFDNet DCDet OpenPCDet FSD
Closest papers by meaning
- VoxelNeXt: Fully Sparse VoxelNet for 3D Object Detection and Tracking (CVPR 2023 · similarity 0.89)
- SAFDNet: A Simple and Effective Network for Fully Sparse 3D Object Detection (CVPR 2024 · similarity 0.89)
- Fully Sparse 3D Object Detection (NeurIPS 2022 · similarity 0.88)
- SBNet: Sparse Blocks Network for Fast Inference (CVPR 2018 · similarity 0.88)
- Robust Single-Stage Fully Sparse 3D Object Detection via Detachable Latent Diffusion (AAAI 2026 · similarity 0.87)
- Voxel or Pillar: Exploring Efficient Point Cloud Representation for 3D Object Detection (AAAI 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.