Research map / Point clouds / Segmentation and detection
LION: Linear Group RNN for 3D Object Detection in Point Clouds
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LION, a window-based 3D object detection backbone that uses linear group RNNs (Mamba, RWKV, RetNet) over large voxel groups, with a 3D spatial feature descriptor and an auto-regressive voxel generation step for sparse point clouds.
Problem
Transformer-based 3D object detectors can only model long-range relations within small voxel groups due to quadratic attention cost, and linear RNNs struggle with spatial modeling and densifying highly sparse point clouds.
Approach
Apply linear group RNN operators to large equal-size groups of voxels (thousands per group), augment them with a 3D sub-manifold conv spatial descriptor to recover local 3D structure, and generate foreground voxels by diffusing high-response voxels and exploiting the RNN's auto-regressive property to fill zero-initialized features.
Key results
- LION-Mamba-L reaches 75.9 mAP / 74.0 mAPH (L1) on Waymo validation, exceeding DSVT-Voxel's 74.0/72.1 (L1), per the main table.
- LION-Mamba achieves 72.1 NDS / 68.0 mAP on nuScenes validation and 73.9 NDS on nuScenes test, beating DSVT's 72.7 NDS on test.
- LION-Mamba reaches 41.5 mAP on Argoverse V2 validation, versus SAFDNet's 39.7, and 66.6 mAP on ONCE, versus CenterPoint's 60.1.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LION-Mamba instead of DSVT | 3D detection · nuScenes test | NDS: 72.7 -> 73.9 | Helped |
| Use LION-Mamba instead of SAFDNet | 3D detection · Argoverse V2 validation | mAP: 39.7 -> 41.5 | Helped |
| Use LION-Mamba instead of CenterPoint | 3D detection · ONCE validation | mAP: 60.1 -> 66.6 | Helped |
| Add large group sizes (ablation) | LION-Mamba · Waymo validation, 20% training data | mAPH L2: 65.8 -> 66.9 | Helped |
| Add the 3D spatial feature descriptor (ablation) | LION-Mamba with large groups · Waymo validation, 20% training data | mAPH L2: 66.9 -> 68.6 | Helped |
| Add voxel generation (ablation) | LION-Mamba with large groups and 3D SFD · Waymo validation, 20% training data | mAPH L2: 68.6 -> 69.3 | Helped |
| Replace the 3D spatial feature descriptor with MLP (ablation) | LION-Mamba · Waymo validation, 20% training data | mAPH L2: 67.6 -> 67.7 | No clear effect |
| Replace the 3D spatial feature descriptor with a linear group RNN (ablation) | LION-Mamba · Waymo validation, 20% training data | mAPH L2: 67.6 -> 67.9 | Helped |
| Use zero-initialized generated features with autoregressive LION blocks instead of sparse convolution (ablation) | LION voxel generation · Waymo validation, 20% training data | mAPH L2: 66.6 -> 69.3 | Helped |
| Initialize generated features with K-NN features instead of zeros (ablation) | LION voxel generation · Waymo validation, 20% training data | mAPH L2: 69.3 -> 68.3 | Hurt |
| Place 3D SFD before voxel merging instead of after (ablation) | LION-Mamba · Waymo validation, 20% training data | mAPH L2: 68.6 -> 69.3 | Helped |
| Increase voxel-generation foreground ratio from 0.2 to 0.5 (ablation) | LION-Mamba · Waymo validation, 20% training data | mAPH L2: 69.3 -> 69.3 | No clear effect |
Surprising findings
- Using different window sizes or group sizes at inference time barely changes performance, suggesting weak dependence on hand-crafted priors.
- Replacing the 3D spatial descriptor with a plain linear group RNN gives only marginal gain (0.3 mAPH/L2), indicating linear group RNNs are poor at local spatial modeling.
Limitations
- Running speed is not as good as transformers because linear RNNs are less efficient for parallel computation.
- Distant objects with sparse points are still missed; authors suggest fusing image features as future work.
- Requires relatively high computing resources for faster running speed.
Experimental setup
Datasets: Waymo Open Dataset nuScenes Argoverse V2 ONCE KITTI
Benchmarks: Waymo Open Dataset nuScenes Argoverse V2 ONCE KITTI
Metrics: mAP mAPH NDS AP
Baselines: DSVT-Voxel DSVT-Pillar HEDNet SAFDNet VoxelNeXt CenterPoint SST SWFormer FSD
Builds on: Mamba RWKV RetNet DSVT FlatFormer CenterPoint
Closest papers by meaning
- JPV-Net: Joint Point-Voxel Representations for Accurate 3D Object Detection (AAAI 2022 · similarity 0.88)
- Towards Learning Group-Equivariant Features for Domain Adaptive 3D Detection (NeurIPS 2024 · similarity 0.88)
- Voxel Mamba: Group-Free State Space Models for Point Cloud based 3D Object Detection (NeurIPS 2024 · similarity 0.87)
- ImOV3D: Learning Open Vocabulary Point Clouds 3D Object Detection from Only 2D Images (NeurIPS 2024 · similarity 0.87)
- SPGroup3D: Superpoint Grouping Network for Indoor 3D Object Detection (AAAI 2024 · similarity 0.86)
- LSK3DNet: Towards Effective and Efficient 3D Perception with Large Sparse Kernels (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.