Research map / Point clouds / Segmentation and detection
SSLFusion: Scale and Space Aligned Latent Fusion Model for Multimodal 3D Object Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SSLFusion, a multi-stage LiDAR-camera 3D detector that aligns scale and space across modalities and uses latent-space cross-modal fusion instead of QKV attention, reporting state-of-the-art KITTI and DENSE results.
Problem
Multimodal (LiDAR-camera) 3D object detection suffers from scale and spatial misalignment between 2D image features and 3D point cloud features, single-stage fusion, and the high cost of QKV cross-attention.
Approach
Fuses image and voxel features at every backbone stage with scale-matched pairing, injects 3D voxel-center coordinates into projected 2D image features via depth embeddings, and performs cross-modal non-local interaction through a small set of latent nodes (LatentGNN-style) with linear complexity rather than QKV attention.
Key results
- On the KITTI test set (car, moderate), SSLFusion reports 84.4 3D AP, a 2.15-point gain over GraphAlign per the abstract and 1.17 over VPFNet per the text.
- On KITTI val (car), 3D AP improves over the Voxel-RCNN baseline by +1.74, +0.42 and +2.50 on easy, moderate and hard levels.
- On DENSE dense-fog split, SSLFusion reaches 78.15 / 76.36 / 71.76 3D AP versus 69.03 / 69.31 / 66.24 for Voxel-RCNN.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SSLFusion instead of GraphAlign | KITTI Car detection · KITTI test, Car moderate | 3D AP: 82.2 -> 84.4 | Helped |
| Use SSLFusion instead of VoxelNeXt | DENSE Car detection · DENSE all weather test, Car moderate | 3D AP: 71.6 -> 79.0 | Helped |
| Use SSLFusion instead of Voxel-RCNN | DENSE Car detection · DENSE dense fog test, Car moderate | 3D AP: 69.3 -> 76.4 | Helped |
| Add scale-aligned fusion (SAF) (ablation) | SSLFusion on the Voxel-RCNN baseline · KITTI val, Car easy | 3D AP: 92.38 -> 92.93 | Helped |
| Add scale-aligned fusion (SAF) (ablation) | SSLFusion on the Voxel-RCNN baseline · KITTI val, Car moderate | 3D AP: 85.29 -> 85.50 | Helped |
| Add 3D-to-2D space alignment (SAM) to SAF (ablation) | SSLFusion on the Voxel-RCNN baseline · KITTI val, Car hard | 3D AP: 83.01 -> 84.75 | Helped |
| Add latent cross-modal fusion (LFM) to SAF and SAM (ablation) | SSLFusion on the Voxel-RCNN baseline · KITTI val, Car hard | 3D AP: 84.75 -> 85.36 | Helped |
| Use latent cross-modal fusion instead of QKV-based attention fusion | Single-stage fusion · KITTI validation | 3D AP: 86.49% -> 86.68% | Helped |
| Use latent cross-modal fusion instead of QKV-based attention fusion | Single-stage fusion · KITTI validation | inference speed: 9.69 FPS -> 10.75 FPS | Helped |
Surprising findings
- Latent cross-modal fusion gives only marginal gains on the moderate KITTI val level (+0.04 3D AP), and SAF gives small gains (+0.21 moderate).
Limitations
- Gains on the KITTI val moderate level are small, with some classes and levels not best (e.g., Ped. moderate lower than the strongest prior method on val cars).
- Evaluation on DENSE relies on re-trained baselines since no official leaderboard exists.
- Fusion details such as the number of latent nodes n and the depth-embedding design are not fully specified in the text.
Experimental setup
Datasets: KITTI DENSE
Benchmarks: KITTI DENSE
Metrics: 3D AP BEV AP
Baselines: Voxel-RCNN PointRCNN PV-RCNN M3DETR Octr MVXNet CLOCs Focals Conv VPFNet CAT-Det HMFI DVF
Builds on: Voxel-RCNN LatentGNN Feature Pyramid Network DeepFusion HFMI
Closest papers by meaning
- CCF: Complementary Collaborative Fusion for Domain Generalized Multi-Modal 3D Object Detection (CVPR 2026 · similarity 0.88)
- STXD: Structural and Temporal Cross-Modal Distillation for Multi-View 3D Object Detection (NeurIPS 2023 · similarity 0.87)
- Cross-Modal Distillation for 2D/3D Multi-Object Discovery from 2D Motion (CVPR 2025 · similarity 0.86)
- DiMSOD: A Diffusion-Based Framework for Multi-Modal Salient Object Detection (AAAI 2025 · similarity 0.86)
- SP3D: Boosting Sparsely-Supervised 3D Object Detection via Accurate Cross-Modal Semantic Prompts (CVPR 2025 · similarity 0.86)
- ImOV3D: Learning Open Vocabulary Point Clouds 3D Object Detection from Only 2D Images (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.