Research map / Adversarial robustness and security / Backdoor and poisoning
BadCLIP: Dual-Embedding Guided Backdoor Attack on Multimodal Contrastive Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BadCLIP, a dual-embedding guided visual-trigger backdoor attack on CLIP that evades DECREE detection and CleanCLIP/fine-tuning defenses while keeping high ASR.
Problem
Existing backdoor attacks on multimodal contrastive learning models such as CLIP are easily neutralized by backdoor detection and clean fine-tuning defenses, so the paper asks whether a backdoor can survive these defenses.
Approach
Optimize a patch-based visual trigger so that triggered images approximate the textual embedding of the target label (textual embedding consistency) and align with real target-class visual features (visual embedding resistance), with poisoned samples chosen as boundary, farthest, and random samples, so that backdoor parameter changes stay small and the backdoor resists clean fine-tuning.
Key results
- BadCLIP achieves 98.81% ASR without defense and 92.50% ASR after FT and 89.60% after CleanCLIP fine-tuning on zero-shot ImageNet-1K, versus e.g. 44.30% for TrojVQA under CleanCLIP.
- In linear probe, BadCLIP reaches 99.14% ASR and 66.40% under CleanCLIP, while baseline attacks stay below 0.1% ASR without defense.
- Under DECREE detection, BadCLIP's PL1-norm is 20506.9 (well above the 0.1 threshold) while baselines fall below 0.1, so it evades detection.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use BadCLIP instead of TrojVQA | CLIP backdoor attack · ImageNet-1K val zero-shot, CleanCLIP | ASR: 44.30 -> 89.60 | Helped |
| Use BadCLIP instead of TrojVQA | CLIP backdoor attack · ImageNet-1K linear probe, CleanCLIP | ASR: 0.08 -> 66.40 | Helped |
| Use BadCLIP instead of TrojVQA | CLIP backdoor attack · ImageNet-1K val zero-shot, cross-domain CleanCLIP (SBU) | ASR: 49.01 -> 87.21 | Helped |
| Use textual consistency loss Lt alone (ablation) | BadCLIP · ImageNet-1K val zero-shot, CleanCLIP | ASR: 44.30 -> 74.47 | Helped |
| Use visual resistance losses Lp_i + Ln_i alone (ablation) | BadCLIP · ImageNet-1K val zero-shot, CleanCLIP | ASR: 44.30 -> 65.24 | Helped |
| Add visual resistance losses to textual consistency loss (ablation) | BadCLIP · ImageNet-1K val zero-shot, CleanCLIP | ASR: 74.47 -> 87.56 | Helped |
| Add poisoned-pair sampling (PPS) to the full loss (ablation) | BadCLIP · ImageNet-1K val zero-shot, CleanCLIP | ASR: 87.56 -> 89.60 | Helped |
| Increase the trigger patch size (ablation) | BadCLIP · ImageNet-1K val zero-shot, no defense and CleanCLIP | ASR | Mixed |
| Increase the number of poisoned samples (ablation) | BadCLIP · ImageNet-1K val zero-shot, no defense and CleanCLIP | ASR | Helped |
Surprising findings
- The ABL-based defense performs poorly against backdoor attacks in the MCL setting, suggesting existing unlearning defenses are limited for MCL.
- Baseline attacks like mmPoison reach near-zero ASR even without defense in the zero-shot setting, while BadCLIP succeeds.
Limitations
- Evaluation is limited to the CLIP model and the banana target label on ImageNet-1K.
- Attacker is assumed to control the pre-training process with white-box access to CLIP parameters and the pre-training data.
Experimental setup
Datasets: CC3M ImageNet-1K SBU Captions
Benchmarks: ImageNet-1K
Metrics: clean accuracy attack success rate PL1-norm
Baselines: BadNet Blended SIG SSBA TrojVQA mmPoison BadEncoder Carlini et al.
Builds on: CLIP CleanCLIP BadEncoder BadNet DECREE
Closest papers by meaning
- Defending Multimodal Backdoored Models by Repulsive Visual Prompt Tuning (NeurIPS 2025 · similarity 0.91)
- Test-Time Multimodal Backdoor Detection by Contrastive Prompting (ICML 2025 · similarity 0.90)
- BadCLIP: Trigger-Aware Prompt Learning for Backdoor Attacks on CLIP (CVPR 2024 · similarity 0.90)
- ToxicTextCLIP: Text-Based Poisoning and Backdoor Attacks on CLIP Pre-training (NeurIPS 2025 · similarity 0.88)
- Better Safe than Sorry: Pre-training CLIP against Targeted Data Poisoning and Backdoor Attacks (ICML 2024 · similarity 0.88)
- Revisiting Backdoor Attacks against Large Vision-Language Models from Domain Shift (CVPR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.