Research map / Adversarial robustness and security / Attacks on text and graphs
Fusion Is Not Enough: Single Modal Attacks on Fusion Models for 3D Object Detection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a camera-only, two-stage adversarial patch attack that identifies image sensitivity distributions and then crafts scene-oriented or object-oriented patches to fool camera-LiDAR fusion 3D detectors on nuScenes.
Problem
Camera-LiDAR fusion models for 3D object detection are assumed robust because multi-sensor redundancy defends against attacks; it is unclear whether attacking only the less important camera modality can subvert them, and existing camera-only patch attacks do not yield deployable patches for fusion m
Approach
A learnable mask-and-perturbation optimization maps which image regions are sensitive to attack; models are classified as object-sensitive or globally sensitive, and each type gets a tailored patch attack (scene-oriented on road, or object-oriented on target vehicle) using 3D-geometry projection and EoT.
Key results
- Scene-oriented patch on the road reduces mAP of BEVFusion-PKU and BEVFusion-MIT from 0.824 to 0.353 on average, with mAP drop above 35%.
- Object-oriented patch reduces target object detection score from 0.728 to 0.156 on average across fusion models and the camera-only BEVFormer.
- Physical-world experiment with replaced front-view images of 30 frames shows significant pedestrian detection score drops with the deployed patch.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply a scene-oriented camera-only patch to static background structures | BEVFusion-PKU and BEVFusion-MIT · NuScenes test set | mAP: 0.824 to 0.353 | Hurt |
| Attach an object-oriented camera-only patch to the target object | Six camera-LiDAR fusion models and BEVFormer · NuScenes test set | target object detection score: 0.728 to 0.156 | Hurt |
| Apply a scene-oriented patch to a non-object road area | DeepInteraction · NuScenes, Scene 2 | pedestrian AP | No clear effect |
| Apply a scene-oriented patch to a non-object road area | BEVFusion-PKU · NuScenes, Scene 2 | pedestrian AP | Hurt |
| Patch an object rather than the road | DeepInteraction and BEVFusion-PKU · NuScenes, Scene 2 | patched-object detection score | Hurt |
| Apply a scene-oriented attack to globally sensitive models rather than object-sensitive models | Camera-LiDAR fusion models · NuScenes test set | mAP | Mixed |
| Attach a camera-only patch to the target object | Other objects in the scene · NuScenes test set | mAP of other objects | No clear effect |
| Reduce camera input information with random noise | Camera-LiDAR fusion models · NuScenes mini | detection performance | Hurt |
| Reduce LiDAR input information with random noise | Camera-LiDAR fusion models · NuScenes mini | detection performance | Hurt |
| Use LiDAR-only input instead of camera-only input | BEVFusion-MIT, BEVFusion-PKU, and UVTR · NuScenes mini | mAP: 0.333 to 0.576; 0.227 to 0.649; 0.362 to 0.609 | Helped |
Surprising findings
- LiDAR-only and ablated LiDAR fusion models outperform camera-only counterparts, yet camera-only attacks still compromise globally sensitive fusion models.
- Object-sensitive fusion models remain robust to scene-oriented patches placed on non-object background such as the road.
Limitations
- Attack effectiveness depends on the relative weighting of camera versus LiDAR in the fusion model.
- Physical-world experiment replaces only front-view images of 30 frames and uses a smartphone camera and printed patch rather than a real vehicle.
Experimental setup
Datasets: nuScenes
Benchmarks: nuScenes
Metrics: mAP average precision detection score
Baselines: BEVFormer one-stage regional patch optimization (camera-only patch attack)
Builds on: Expectation over Transformation GradCAM ScoreCAM BEVFormer
Closest papers by meaning
- COMMIT: Certifying Robustness of Multi-Sensor Fusion Systems Against Semantic Attacks (AAAI 2025 · similarity 0.88)
- NumbOD: A Spatial-Frequency Fusion Attack Against Object Detectors (AAAI 2025 · similarity 0.86)
- A²RNet: Adversarial Attack Resilient Network for Robust Infrared and Visible Image Fusion (AAAI 2025 · similarity 0.86)
- Resilient Sensor Fusion Under Adverse Sensor Failures via Multi-Modal Expert Fusion (CVPR 2025 · similarity 0.85)
- A New Adversarial Perspective for LiDAR-based 3D Object Detection (AAAI 2025 · similarity 0.84)
- SimDistill: Simulated Multi-Modal Distillation for BEV 3D Object Detection (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.