Research map / Detection and segmentation / 3D detection and driving
NBA3D: Neighbor-Based Confidence Adjustment for 3D Rare Object Detection Using LiDAR
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes NBA3D, a GNN-based post-hoc module that reassigns confidence scores of rare-class LiDAR 3D detections using their neighboring prediction boxes and CLIP text embeddings, improving rare-class AP on nuScenes long-tail without extra sensors.
Problem
LiDAR-based 3D object detectors trained on long-tail nuScenes perform much worse on rare classes (e.g., stroller, personal mobility) than dominant classes, and prior remedies rely on camera fusion and architecture changes.
Approach
Build a graph around each rare-class prediction box from neighboring detections (box info, confidence, CLIP class text embedding), use a GNN (TPDGNN) to predict true-positive score and distance to nearest rare GT, and average these with the original confidence to rescore the box, requiring only detector outputs.
Key results
- On SphereFormer, NBA3D raises Stroller AP from 3.8 to 20.2 and rare-class mAP from 5.7 to 10.8 with no change to Many/Medium mAP.
- On VoxelNeXt, Personal Mobility AP improves from 3.6 to 11.3 and rare mAP from 8.2 to 12.0.
- NBA3D adds about 0.99 ms total overhead (0.31 ms graph generation, 0.61 ms normalization, 0.05 ms GNN inference).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply NBA3D confidence adjustment | SphereFormer · nuScenes val, Rare | mAP: 5.7 -> 10.8 | Helped |
| Apply NBA3D confidence adjustment | LinK · nuScenes val, Rare | mAP: 6.4 -> 9.4 | Helped |
| Apply NBA3D confidence adjustment | VoxelNeXt · nuScenes val, Rare | mAP: 8.2 -> 12.0 | Helped |
| Apply NBA3D confidence adjustment | PillarNet · nuScenes val, Rare | mAP: 6.4 -> 7.6 | Helped |
| Add one-hot class, confidence, and box information as node features (ablation) | NBA3D on SphereFormer · nuScenes val, Rare | mAP: 5.7 -> 7.0 | Helped |
| Add class text embeddings to node features (ablation) | NBA3D on SphereFormer · nuScenes val, Rare | mAP: 7.0 -> 10.8 | Helped |
| Limit the graph to a maximum of 5 nearest neighbors (ablation) | NBA3D on SphereFormer · nuScenes val, Rare | mAP: 10.8 -> 7.1 | Hurt |
| Limit the graph to a maximum of 10 nearest neighbors (ablation) | NBA3D on SphereFormer · nuScenes val, Rare | mAP: 10.8 -> 9.1 | Hurt |
| Remove the ground-truth distance loss (ablation) | NBA3D on SphereFormer · nuScenes val, Rare | mAP: 10.8 -> 10.2 | Hurt |
| Remove the true-positive classification loss (ablation) | NBA3D on SphereFormer · nuScenes val, Rare | mAP: 10.8 -> 5.7 | Hurt |
Surprising findings
- Removing the distance-to-GT prediction loss and averaging only true-positive score with original confidence still helps, but the full combined loss performs best.
- Limiting neighbors to 5 or 10 nearest boxes underperforms using all boxes within the 4 m radius.
Limitations
- Relies on detector predictions for graph construction, so it cannot recover rare objects the base detector misses entirely.
- Gains are uneven across rare classes (e.g., Emergency Vehicle improvements are small) and absolute rare-class AP remains low.
Experimental setup
Datasets: nuScenes long-tail variant (18 classes)
Benchmarks: nuScenes long-tail validation set (18 classes)
Metrics: average precision (AP) averaged over 0.5, 1, 2, 4 m center d mAP for Many/Medium/Rare groups
Baselines: SphereFormer LinK VoxelNeXt PillarNet Peri et al. group-free head with camera fusion
Builds on: CLIP Peri et al. long-tail 3D detection benchmark NNConv GAT sigmoid focal loss
Closest papers by meaning
- An Empirical Study of the Generalization Ability of Lidar 3D Object Detectors to Unseen Domains (CVPR 2024 · similarity 0.88)
- LSK3DNet: Towards Effective and Efficient 3D Perception with Large Sparse Kernels (CVPR 2024 · similarity 0.87)
- SAFDNet: A Simple and Effective Network for Fully Sparse 3D Object Detection (CVPR 2024 · similarity 0.87)
- Training an Open-Vocabulary Monocular 3D Detection Model without 3D Data (NeurIPS 2024 · similarity 0.87)
- STONE: A Submodular Optimization Framework for Active 3D Object Detection (NeurIPS 2024 · similarity 0.87)
- A-Teacher: Asymmetric Network for 3D Semi-Supervised Object Detection (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.