Research map / Detection and segmentation / 3D detection and driving
ViKIENet: Towards Efficient 3D Object Detection with Virtual Key Instance Enhanced Network
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ViKIENet, a LiDAR-camera 3D detector that fuses only semantically selected virtual key instances (not all virtual points) with LiDAR via VIFF and VIRA modules, plus a rotation-equivariant variant ViKIENet-R.
Problem
LiDAR-only 3D detection suffers from sparse points and weak semantics; virtual-point fusion methods are computationally expensive due to dense virtual points and noisy from inaccurate depth completion.
Approach
Select only semantically relevant instance pixels from an image segmentation mask and back-project them into virtual key instances (VKIs) with class scores; fuse VKIs with LiDAR features in BEV and RoI stages (VIFF) and refine VKI features using precise LiDAR features via cross-attention (VIRA).
Key results
- On KITTI val car 3D AP, ViKIENet improves over Voxel R-CNN by about 3.2% at easy, moderate, and hard levels, running at 22.7 FPS on KITTI test.
- On KITTI test car 3D AP (R40), ViKIENet-R reaches 91.20/86.04/81.18 at 15.0 FPS, with ViKIENet ranking first on the KITTI car detection and orientation leaderboards at submission time.
- On JRDB validation, ViKIENet achieves 53.16 3D mAP vs 51.03 for VirConv-L while running at 19.2 FPS vs 8.6 FPS.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ViKIENet instead of Voxel-RCNN | KITTI test car detection · KITTI test, car | 3D AP (R40), moderate: 81.62 -> 84.96 | Helped |
| Use ViKIENet instead of VirConv-T | KITTI val car detection · KITTI val, car | 3D AP (R40), easy: 94.58 -> 95.58 | Helped |
| Use ViKIENet instead of VirConv-T | KITTI val car detection · KITTI val, car | 3D AP (R40), moderate: 89.87 -> 88.49 | Hurt |
| Use ViKIENet instead of VirConv-L | JRDB detection · JRDB val | 3D mAP: 51.03 -> 53.16 | Helped |
| Use ViKIENet + MVP instead of VirConv + MVP | nuScenes detection · nuScenes val | mAP: 66.73 -> 68.72 | Helped |
| Add BEV fusion (ablation) | VIFF on KITTI car detection · KITTI val, car | AP (R40), hard: 83.20 -> 84.71 | Helped |
| Add RoI fusion (ablation) | VIFF on KITTI car detection · KITTI val, car | AP (R40), moderate: 85.41 -> 87.75 | Helped |
| Add BEV fusion and RoI fusion (ablation) | VIFF on KITTI car detection · KITTI val, car | AP (R40), moderate: 85.41 -> 88.09 | Helped |
| Add VIRA to VIFF with BEV and RoI fusion (ablation) | ViKIENet on KITTI car detection · KITTI val, car | AP (R40), moderate: 88.09 -> 88.49 | Helped |
| Add semantic VKIs with SKIS and VIFF to VirConv-L (ablation) | VirConv-L on KITTI car detection · KITTI val, car | 3D AP (R40), easy: 93.18 -> 95.19 | Helped |
Surprising findings
- ViKIENet-R does not beat VirConv-T on the KITTI validation set for car 3D AP at easy level, showing the rotation-equivariant variant trades some accuracy for speed.
Limitations
- Relies on segmentation and depth completion quality; false negatives from segmentation can leave VKI features empty.
- ViKIENet-R is slower than ViKIENet and does not beat VirConv-T on all metrics.
Experimental setup
Datasets: KITTI JRDB nuScenes
Benchmarks: KITTI JRDB nuScenes
Metrics: 3D AP (R40) BEV AP AOS mAP FPS
Baselines: Voxel-RCNN VirConv-L VirConv-T SFD PV-RCNN LoGoNet Focals Conv TED MVP TSSTDet BtcDet VPFNet
Builds on: Voxel R-CNN VirConv SFD PointPainting PENet
Closest papers by meaning
- Virtual Sparse Convolution for Multimodal 3D Object Detection (CVPR 2023 · similarity 0.91)
- Multimodal Virtual Point 3D Detection (NeurIPS 2021 · similarity 0.90)
- VoxelNet: End-to-End Learning for Point Cloud Based 3D Object Detection (CVPR 2018 · similarity 0.89)
- SAFDNet: A Simple and Effective Network for Fully Sparse 3D Object Detection (CVPR 2024 · similarity 0.87)
- Beyond Virtual Points: Depth-Enhanced LiDAR-only 3D Object Detection with Semi-Supervised Learning (Student Abstract) (AAAI 2025 · similarity 0.87)
- DI-V2X: Learning Domain-Invariant Representation for Vehicle-Infrastructure Collaborative 3D Object Detection (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.