Research map / Adversarial robustness and security / Backdoor and poisoning
Attack on Prompt: Backdoor Attack in Prompt-Based Continual Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AOP, a black-box, clean-label backdoor attack on prompt-based class-incremental learners that uses a surrogate dataset, static-dynamic trigger optimization, and BCE loss to create transferable, resilient, and authentic triggers.
Problem
Backdoor attacks against prompt-based continual learning in multi-data-supplier settings, where an attacker controls only a small fraction of the data (one class) and the victim learns incrementally without stored memory.
Approach
Exploit prompt selection and visual prompting's label-mapping property so triggers transfer from a surrogate dataset to unknown data; alternate static and dynamic surrogate stages so the trigger survives continual updates; and replace softmax cross-entropy with sigmoid BCE so the trigger does not degenerate into adversarial noise.</ättribute>
Key results
- AOP reaches about 99.9% ASR on L2P, DualPrompt and PGP variants on 5-Split-CUB200 with TinyImageNet or CIFAR100 surrogates, with clean accuracy changes under 0.5 points.
- On 10-Split-ImageNet-R, BCE-optimized triggers give 0.00% top-1 ASR on clean non-attacked L2P, versus 74.18% for CE-optimized triggers.
- AOP maintains high ASR across tasks while Narcissus degrades due to forgetting in the CUB200 task-by-task comparison (no exact numbers given in text).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use sigmoid BCE instead of softmax CE to optimize the trigger (ablation) | AOP on L2P · 10-Split-ImageNet-R, clean model tested on triggered samples | Top-1 ASR: 74.18 -> 0.00 | Helped |
| use sigmoid BCE instead of softmax CE to optimize the trigger (ablation) | AOP on DualPrompt · 5-Split-ImageNet-R, clean model tested on triggered samples | Top-1 ASR: 34.18 -> 0.00 | Helped |
| use sigmoid BCE instead of softmax CE to optimize the trigger (ablation) | AOP on DualPrompt · 10-Split-ImageNet-R, clean model tested on triggered samples | Top-1 ASR: 42.85 -> 0.00 | Helped |
| use sigmoid BCE instead of softmax CE to optimize the trigger (ablation) | AOP on DualPrompt · 20-Split-ImageNet-R, clean model tested on triggered samples | Top-1 ASR: 96.93 -> 0.00 | Helped |
| use TinyImageNet rather than CIFAR100 as the surrogate dataset | AOP on L2P · 5-Split-CUB200 | ASR: 99.96 \pm 0.02 -> 99.99 \pm 0.02 | No clear effect |
| increase the task sequence from 5 to 20 splits | AOP on L2P · ImageNet-R | ASR: 99.76 \pm 0.10 -> 98.24 \pm 0.21 | Hurt |
Surprising findings
- Triggers optimized with softmax cross-entropy transform into adversarial-like noise that misclassifies clean models, while sigmoid BCE eliminates this effect.
- FST defense mitigates the attack (ASR 0%) but causes large clean accuracy drops due to classifier reinitialization.
Limitations
- Some competition between target and other classes is needed; relying solely on BCE may hurt performance.
- Defenses evaluated may not be optimal for continual learning scenarios.
- ASR decays for long task sequences (20-split) due to catastrophic forgetting.
Experimental setup
Datasets: ImageNet-R CUB200 TinyImageNet CIFAR100 ImageNet-1K iBOT-1K
Benchmarks: Split-ImageNet-R Split-CUB200
Metrics: attack success rate average accuracy
Baselines: Narcissus Label Consistent (LC)
Builds on: L2P DualPrompt Narcissus PGP
Closest papers by meaning
- BadPrompt: Backdoor Attacks on Continuous Prompts (NeurIPS 2022 · similarity 0.88)
- Demystifying Poisoning Backdoor Attacks from a Statistical Perspective (ICLR 2024 · similarity 0.86)
- BrainWash: A Poisoning Attack to Forget in Continual Learning (CVPR 2024 · similarity 0.86)
- Rethinking Backdoor Attacks on Dataset Distillation: A Kernel Method Perspective (ICLR 2024 · similarity 0.85)
- Unveiling and Mitigating Backdoor Vulnerabilities based on Unlearning Weight Changes and Backdoor Activeness (NeurIPS 2024 · similarity 0.85)
- Clean Data Can Still Carry Backdoors: Support-Persistent Backdoors for Model Reuse (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.