Research map / Point clouds / Segmentation and detection
Voxel or Pillar: Exploring Efficient Point Cloud Representation for 3D Object Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes VPF, a hybrid voxel-pillar backbone with a Sparse Fusion Layer for bidirectional voxel-pillar feature interaction, integrated into dense and sparse 3D detectors for LiDAR detection.
Problem
Grid-based LiDAR 3D detectors use either voxels or pillars, and their distinct vertical representation limitations (voxels: restricted receptive field from submanifold conv; pillars: information loss in dense areas) are underexplored.
Approach
A dual-branch sparse voxel-pillar encoder (3D sparse conv for voxels, 2D sparse conv for pillars, with matched horizontal downsampling) plus a Sparse Fusion Layer that max-pools voxels vertically into pillars and broadcasts pillars back to voxels, fused by element-wise summation after each stage.
Key results
- On the WOD val set, VPF_de achieves 73.9/71.7 LEVEL 2 mAP/mAPH single-stage, outperforming DSVT (73.2/71.0) and CenterPoint (71.7/69.4).
- On the nuScenes test set, VPF_de reaches 67.0 mAP and 72.7 NDS single model, rising to 73.8 NDS with flip test.
- Sparse Fusion Layer adds 1.7 mAPH over naive voxel-pillar encoder with DFN on WOD val (68.0 to 69.7), at a latency of 75 ms versus 69 ms.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| combine voxel and pillar representations (ablation) | VPF_de · Waymo Open Dataset val, LEVEL 2 | mAPH: 67.5/66.3 -> 68.3 | Helped |
| add SFL to voxel-pillar encoding with DFN (ablation) | VPF_de · Waymo Open Dataset val, LEVEL 2 | mAPH: 68.0 -> 69.7 | Helped |
| use bidirectional rather than one-way voxel-pillar interaction (ablation) | VPF_de · Waymo Open Dataset val, LEVEL 2 | mAPH: 69.2; 68.1 -> 69.7 | Helped |
| add a vertical residual to CenterPoint (ablation) | Scaled CenterPoint · Waymo Open Dataset val, LEVEL 2 | mAPH: 68.8 -> 69.0 | Helped |
| replace the hybrid branches with two voxel branches (ablation) | VPF_de · Waymo Open Dataset val, LEVEL 2 | mAPH: 69.7 -> 68.2 | Hurt |
| replace the hybrid branches with two pillar branches (ablation) | VPF_de · Waymo Open Dataset val, LEVEL 2 | mAPH: 69.7 -> 66.9 | Hurt |
| remove SFL (ablation) | VPF_sp · Waymo Open Dataset val, LEVEL 2 | mAPH: 70.1 -> 70.9 | Hurt |
| use voxel-only instead of the fused dense neck (ablation) | VPF_de · Waymo Open Dataset val, LEVEL 2 | mAPH: 67.7 -> 69.7 | Hurt |
| use pillar-only instead of the fused dense neck (ablation) | VPF_de · Waymo Open Dataset val, LEVEL 2 | mAPH: 69.1 -> 69.7 | Hurt |
| replace dense-neck summation with concatenation (ablation) | VPF_de · Waymo Open Dataset val, LEVEL 2 | mAPH: 69.7 -> 69.6 | No clear effect |
| replace dense-neck summation with adaptive fusion (ablation) | VPF_de · Waymo Open Dataset val, LEVEL 2 | mAPH: 69.7 -> 69.2 | No clear effect |
| use flipping test | VPF_de · nuScenes test | NDS: 72.7 -> 73.8 | Helped |
Surprising findings
- Naive dual-branch voxel+pillar with late fusion yields comparable results to the ensemble while being more efficient, and voxel-only or pillar-only models with matched parameter counts underperform the hybrid by a wide margin.
Limitations
- Relies on consistent BEV occupancy, requiring equal horizontal partitioning for voxels and pillars.
- Failure cases occur when foreground and background points intertwine or points are extremely sparse.
- Sparse variant is weaker than dense variant on vehicles due to harder center estimation.
Experimental setup
Datasets: Waymo Open Dataset nuScenes
Benchmarks: Waymo Open Dataset nuScenes
Metrics: AP APH mAP mAPH NDS latency
Baselines: CenterPoint PillarNet VoxelNeXt GD-MAE DSVT PointPillars FSD LinK
Builds on: VoxelNet SECOND PillarNet CenterPoint VoxelNeXt
Closest papers by meaning
- VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection (CVPR 2018 · similarity 0.92)
- PillarNeXt: Rethinking Network Designs for 3D Object Detection in LiDAR Point Clouds (CVPR 2023 · similarity 0.92)
- MsSVT: Mixed-scale Sparse Voxel Transformer for 3D Object Detection on Point Clouds (NeurIPS 2022 · similarity 0.92)
- JPV-Net: Joint Point-Voxel Representations for Accurate 3D Object Detection (AAAI 2022 · similarity 0.92)
- VoxelNeXt: Fully Sparse VoxelNet for 3D Object Detection and Tracking (CVPR 2023 · similarity 0.92)
- Semi-supervised 3D Object Detection with PatchTeacher and PillarMix (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.