Research map / Adversarial robustness and security / Backdoor and poisoning
Influencer Backdoor Attack on Semantic Segmentation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Influencer Backdoor Attack (IBA) on semantic segmentation, where a trigger placed on non-victim pixels causes all victim-class pixels to be misclassified, with Nearest Neighbor Injection and Pixel Random Labeling to improve attack success.
Problem
Backdoor attacks have been studied mainly for classification; segmentation models classify every pixel, and existing attacks do not target a whole victim class via triggers placed on other objects at uncontrolled positions.
Approach
Inject a printable trigger on non-victim pixels so that context aggregation in segmentation models misclassifies all victim-class pixels; NNI places the trigger nearest the victim pixels, and PRL randomly relabels non-victim pixels to other classes present in the same image so the model learns global context, making the attack work at distant trigger positions.
Key results
- On Cityscapes DeepLabV3 at 5% poisoning, PRL reaches 92.46% ASR vs 62.13% for baseline IBA and 82.45% for NNI, with CBA maintained around 73%.
- At trigger distance 120-150 pixels with 5% poisoning, PRL keeps 90.75% ASR while NNI drops to 45.62% and baseline to 54.74%.
- Under fine-tuning defense with 10% clean data, NNI retains 55.08% ASR vs 7.70% for baseline IBA.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Place the trigger nearest to victim pixels instead of randomly | Baseline IBA on DeepLabV3 · Cityscapes poisoned test, 5% poisoning | ASR: 62.13 -> 82.45 | Helped |
| Randomly relabel non-victim pixels with classes present in the same image | Baseline IBA on DeepLabV3 · Cityscapes poisoned test, 5% poisoning | ASR: 62.13 -> 92.46 | Helped |
| Combine NNI and PRL (ablation) | IBA on DeepLabV3 · Cityscapes poisoned test, 5% poisoning, trigger distance 0–60 | ASR: 82.45 -> 94.18 | Helped |
| Combine NNI and PRL (ablation) | IBA on DeepLabV3 · Cityscapes poisoned test, 5% poisoning, trigger distance 120–150 | ASR: 45.62 -> 46.28 | No clear effect |
| Use PRL rather than NNI | IBA on DeepLabV3 · Cityscapes poisoned test, 5% poisoning, trigger distance 120–150 | ASR: 45.62 -> 90.75 | Helped |
| Increase trigger size from 55*55 to 105*105 with random placement (ablation) | PRL on DeepLabV3 · Cityscapes trigger-size ablation | ASR: 73.21 -> 93.57 | Helped |
| Increase trigger size from 55*55 to 105*105 with random placement (ablation) | PRL on DeepLabV3 · Cityscapes trigger-size ablation | PBA: 71.37 -> 65.23 | Hurt |
| Increase trigger size from 55*55 to 105*105 with random placement (ablation) | PRL on DeepLabV3 · Cityscapes trigger-size ablation | CBA: 73.37 -> 68.24 | Hurt |
| Randomly relabel pixels using classes in the same image rather than null labels, a fixed class, or classes from the whole dataset (ablation) | PRL on Cityscapes · Cityscapes poisoned test | ASR | Helped |
| Allow the trigger to overlap pixels from multiple classes (ablation) | IBA on DeepLabV3 · Cityscapes poisoned test, 5% poisoning | ASR: 62.13 -> 61.98 | No clear effect |
| Allow the trigger to overlap victim pixels (ablation) | IBA on DeepLabV3 · Cityscapes poisoned test, 5% poisoning | ASR: 62.13 -> 61.81 | No clear effect |
| Use a black line on top instead of the Hello Kitty trigger (ablation) | Baseline IBA on DeepLabV3 · Cityscapes poisoned test, 5% poisoning | ASR: 62.13 -> 35.36 | Hurt |
Surprising findings
- Combining NNI and PRL does not substantially improve ASR, and the combination's ASR drops with distance like NNI alone.
- Trigger placement overlapping victim pixels or multiple classes yields similar ASR to the constrained setting.
Limitations
- Evaluated only on VOC and Cityscapes with three architectures; generalization to broader real-world conditions is not established.
- Defenses tested are only simple fine-tuning and pruning; adaptation of other defenses is out of scope.
- Real-world experiment uses a single printed trigger on a large sheet and a fixed setup.
Experimental setup
Datasets: PASCAL VOC 2012 Cityscapes
Benchmarks: PASCAL VOC 2012 Cityscapes
Metrics: Attack Success Rate (ASR) Poisoned Benign Accuracy (PBA) Clean Benign Accuracy (CBA) mIoU
Baselines: Baseline IBA OGBA Black-line trigger attack (prior segmentation backdoor) Fine-tuning defense Pruning defense
Builds on: BadNets
Closest papers by meaning
- A Closer Look at Backdoor Attacks on CLIP (ICML 2025 · similarity 0.85)
- Semantic Shield: Defending Vision-Language Models Against Backdooring and Poisoning via Fine-grained Knowledge Alignment (CVPR 2024 · similarity 0.85)
- Attention-Imperceptible Backdoor Attacks on Vision Transformers (AAAI 2025 · similarity 0.85)
- BaDExpert: Extracting Backdoor Functionality for Accurate Backdoor Input Detection (ICLR 2024 · similarity 0.84)
- A Theoretical Analysis of Backdoor Poisoning Attacks in Convolutional Neural Networks (ICML 2024 · similarity 0.84)
- Data Free Backdoor Attacks (NeurIPS 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.