Research map / Adversarial robustness and security / Attacks on text and graphs
Seeing is Not Believing: Adversarial Natural Object Optimization for Hard-Label 3D Scene Attacks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes UAO2, a hard-label black-box attack that places a realistic real-world object trigger in 3D scenes and optimizes it with weighted Monte Carlo gradient estimation to mislocalize targets in 3D visual grounding models.
Problem
Hard-label adversarial attacks on scene-based 3D models (3D visual grounding), where attackers only see predicted outputs and existing point-cloud attacks target single-object classifiers and are not suited to complex scenes.
Approach
Inject a universal realistic object trigger (from ScanObjectNN), placed on a Hough-detected plane near the target object, and optimize it with a weighted Monte Carlo gradient estimator that up-weights noise directions causing mislocalization, using only query access.
Key results
- On ScanRefer, the attack reduces EDA's Overall Acc@0.25 from 85.76 (unique) and 49.13 (multiple) to 20.61 and 15.38 respectively.
- On Sr3D, the attack lowers EDA's overall accuracy from 68.5 to 23.0 (Acc@0.25 scale), and 3D-SPS from 62.6 to 19.1.
- Defenses (sampling, fine-tuning, anomaly detection) only partially recover performance, e.g., EDA Acc@0.25 with fine-tuning rises to 34.64 versus 15.96 undefended.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the UAO² universal adversarial object trigger | EDA · ScanRefer, Unique | Acc@0.25: 85.76 -> 20.61 | Helped |
| Add the UAO² universal adversarial object trigger | EDA · ScanRefer, Multiple | Acc@0.25: 49.13 -> 15.38 | Helped |
| Add the UAO² universal adversarial object trigger | ViL3DRel · Nr3D, Overall | accuracy: 64.4 -> 21.3 | Helped |
| Add the UAO² universal adversarial object trigger | ViL3DRel · Sr3D, Overall | accuracy: 72.8 -> 24.8 | Helped |
| Set object trigger size λ to 0.1 instead of 0.2 (ablation) | UAO² on 3D-SPS · ScanRefer | Acc@0.25: 15.05 -> 18.34 | Hurt |
| Set object trigger size λ to 0.5 instead of 0.2 (ablation) | UAO² on 3D-SPS · ScanRefer | Acc@0.25: 15.05 -> 14.72 | Helped |
| Set IoU threshold γ to 0.5 instead of 0.6 (ablation) | UAO² on 3D-SPS · ScanRefer | Acc@0.25: 15.05 -> 17.29 | Hurt |
| Set IoU threshold γ to 0.7 instead of 0.6 (ablation) | UAO² on 3D-SPS · ScanRefer | Acc@0.25: 15.05 -> 15.92 | Hurt |
| Use 100 optimization samples instead of 200 (ablation) | UAO² on 3D-SPS · ScanRefer | Acc@0.25: 15.05 -> 24.28 | Hurt |
| Use 300 optimization samples instead of 200 (ablation) | UAO² on 3D-SPS · ScanRefer | Acc@0.25: 15.05 -> 14.69 | Helped |
Surprising findings
- Fine-tuning defense recovers accuracy only to about 34.6% while the attack still substantially degrades models, and the attack is reported as insensitive to trigger object type.
Limitations
- Requires about 200 scene samples and 100 queries per gradient estimate, so query cost is substantial.
- Evaluated only on 3D grounding task and three indoor datasets.
Experimental setup
Datasets: ScanRefer Nr3D Sr3D ScanObjectNN
Benchmarks: ScanRefer Nr3D Sr3D
Metrics: Acc@0.25 Acc@0.5 Chamfer distance Hausdorff distance perturbation norm running time memory usage
Baselines: ScanRefer ReferIt3D SAT 3DTrans. 3D-SPS ViL3DRel EDA
Builds on: Monte Carlo gradient estimation Hough transform ScanObjectNN
Closest papers by meaning
- SABER: Spatially Consistent 3D Universal Adversarial Objects for BEV Detectors (CVPR 2026 · similarity 0.89)
- Robust Adversarial Objects against Deep Learning Models (AAAI 2020 · similarity 0.89)
- Hide in Thicket: Generating Imperceptible and Rational Adversarial Perturbations on 3D Point Clouds (CVPR 2024 · similarity 0.89)
- Explicitly Perceiving and Preserving the Local Geometric Structures for 3D Point Cloud Attack (AAAI 2024 · similarity 0.87)
- Towards Transferable Targeted 3D Adversarial Attack in the Physical World (CVPR 2024 · similarity 0.86)
- Manifold Constraints for Imperceptible Adversarial Attacks on Point Clouds (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.