Research map / Adversarial robustness and security / Attacks on text and graphs
Saliuitl: Ensemble Salience Guided Recovery of Adversarial Patches against CNNs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Saliuitl, a two-stage adversarial patch defense that first detects attacks via attributes of an ensemble of binarized saliency feature maps, then recovers by inpainting regions of salient neurons across thresholds.
Problem
Recovering clean predictions of CNN-based object detectors and image classifiers under adversarial patch attacks, when the number, shape, and size of patches are unknown and attacks may not be present, without relying on fixed saliency thresholds.
Approach
Detect attacks explicitly by observing how attributes (active neuron count, DBSCAN cluster statistics) of binarized feature maps evolve across an ensemble of saliency thresholds, fed to a 1D CNN detector; then recover only flagged inputs by iteratively inpainting clusters of salient neurons at decreasing thresholds.
Key results
- On image classification, Saliuitl achieves recovery rates up to 97.81 percentage points higher than recent defenses at the same lost prediction rate (per abstract).
- On object detection, recovery rate is up to 42.63 percentage points higher at the same lost prediction rate (per abstract).
- On CIFAR-10 single-patch attacks, Saliuitl reaches recovery rate 0.9738 with lost prediction rate 0.0008, and outperforms all non-certifiable baselines in Table 1 except Themis on ImageNet single patch.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Saliuitl instead of Object Seeker | YOLOv2 · INRIA multi-object patches | recovery rate: 0.4190 -> 0.4749 | Helped |
| Use Saliuitl instead of Object Seeker | YOLOv2 · VOC single patch | recovery rate: 0.6483 -> 0.5404 | Hurt |
| Use Saliuitl instead of Themis | ResNet-50 · ImageNet single patch | recovery rate: 0.9286 -> 0.8869 | Hurt |
| Use Saliuitl instead of NutNet | ResNet-50 · CIFAR-10 single patch | recovery rate: 0.9641 -> 0.9738 | Helped |
| Use Saliuitl instead of NutNet | ResNet-50 · CIFAR-10 double patches | recovery rate: 0.9617 -> 0.9789 | Helped |
| Use Saliuitl instead of Object Seeker | YOLOv2 · INRIA multi-object patches | adversarial nmAP: 0.2485 -> 0.4737 | Helped |
| Use Saliuitl instead of Object Seeker | YOLOv2 · VOC multi-object patches | adversarial nmAP: 0.2238 -> 0.3563 | Helped |
Surprising findings
- Themis with a known patch-size bound outperforms Saliuitl for single-patch attacks on ImageNet, and is the only method to do so.
- Saliuitl's performance drops notably for triangular (non-rectangular) patches, which the authors attribute to square CNN kernels.
Limitations
- Performance degrades for non-rectangular (triangular) patches.
- Cost grows linearly with ensemble size, which can exceed certifiable defenses for large ensembles.
- Requires tuning of detection threshold alpha* and per-scenario best results reported.
Experimental setup
Datasets: INRIA Pascal VOC ImageNet CIFAR-10
Benchmarks: INRIA Pascal VOC ImageNet CIFAR-10
Metrics: recovery rate lost prediction rate inflicted attack rate adversarial nmAP clean nmAP attack detection accuracy computation time
Baselines: NutNet PAD Jedi Themis Themis-50 Object Seeker Patch Cleanser FNS
Builds on: Themis Jedi PAD NutNet Patch Cleanser Object Seeker
Closest papers by meaning
- CAMERAS: Enhanced Resolution and Sanity Preserving Class Activation Mapping for Image Saliency (CVPR 2021 · similarity 0.85)
- Pixel Is All You Need: Adversarial Trajectory-Ensemble Active Learning for Salient Object Detection (AAAI 2023 · similarity 0.85)
- SaliencyMix: A Saliency Guided Data Augmentation Strategy for Better Regularization (ICLR 2021 · similarity 0.85)
- A Unified, Resilient, and Explainable Adversarial Patch Detector (CVPR 2025 · similarity 0.85)
- No Pains, More Gains: Recycling Sub-Salient Patches for Efficient High-Resolution Image Recognition (CVPR 2025 · similarity 0.84)
- NAPGuard: Towards Detecting Naturalistic Adversarial Patches (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.