Research map / Adversarial robustness and security / Backdoor and poisoning
CleanCLIP: Mitigating Data Poisoning Attacks in Multimodal Contrastive Learning
Hritik Bansal, Nishad Singhi, Yu Yang, Fan Yin, Aditya Grover, Kai-Wei Chang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| finetune poisoned CLIP with CleanCLIP (multimodal + self-supervised loss) on clean pairs | poisoned CLIP (CC3M) · ImageNet-1K val, BadNet | attack success rate (%): 99.94 -> 10.46 | Improved |
| finetune with multimodal contrastive loss only (ablation) | poisoned CLIP (CC3M) · ImageNet-1K val, BadNet | attack success rate (%): 99.94 -> 99.8 | No clear effect |
| finetune with self-supervised loss only (ablation) | poisoned CLIP (CC3M) · ImageNet-1K val, BadNet | ASR and clean accuracy: ASR 99.94 -> 0.9; CA 19.06 -> 13.05 | Mixed |
| add self-supervised loss during pretraining on poisoned data | CLIP pretraining · ImageNet-1K val, 4 backdoor attacks | attack success rate (%): BadNet 99.94 -> 90.72; Blended 99.45 -> 99.16 | No clear effect |
| add multimodal anti-backdoor learning (ABL) unlearning during pretraining | CLIP pretraining · ImageNet-1K val, 4 backdoor attacks | attack success rate (%): BadNet 99.94 -> 99.89 | No clear effect |
| apply CleanCLIP to a poisoned CLIP-400M model | CLIP RN-50 400M · ImageNet-1K val, BadNet | ASR and zero-shot accuracy: ASR 94.6% -> 17%; accuracy 59.6% -> 57% | Improved |
| supervised finetuning of vision encoder on 50K labeled ImageNet images | poisoned CLIP (CC3M) · ImageNet-1K val, four attacks | attack success rate (%): BadNet 99.94 -> 0; clean accuracy 19.06 -> 40.86 | Improved |
| increase self-supervision weight lambda2 from 0.5 to 8 (ablation) | CleanCLIP · Blended attack | attack success rate (%) | Improved |
| use MSCOCO or SBU Captions instead of CC3M as the clean finetuning data (ablation) | CleanCLIP · 4 attacks average | ASR and clean accuracy (%): ASR 95.53 -> 9.76 (MSCOCO), 6.92 (SBU); CA 18.88 -> 15.09, 15.11 | Mixed |
| increase CleanCLIP finetuning set from 10K to 100K pairs (ablation) | CleanCLIP · 4 attacks average | attack success rate (%): 21.65 -> 7.21 | Improved |
| increase number of BadNet poisons from 75 to 1500 (ablation) | CleanCLIP defense · BadNet, CC3M | attack success rate (%): 2.38 -> 7.7 | Hurt |
| increase pretraining data size from 500K to 3M (lower poison ratio) (ablation) | CleanCLIP defense · BadNet, 1500 poisons | attack success rate (%): 24.66 -> 7.7 | Improved |
Later papers that compared with this method
2 later accepted papers report CleanCLIP as a baseline in their results tables; 2 of them report a better number on most of the results they share with it.
- A Closer Look at Backdoor Attacks on CLIP (ICML 2025 · better on 10 of 10 shared results)
- Defending Multimodal Backdoored Models by Repulsive Visual Prompt Tuning (NeurIPS 2025 · better on 4 of 4 shared results)
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- BadCLIP: Dual-Embedding Guided Backdoor Attack on Multimodal Contrastive Learning (CVPR 2024 · similarity 0.91)
- Better Safe than Sorry: Pre-training CLIP against Targeted Data Poisoning and Backdoor Attacks (ICML 2024 · similarity 0.91)
- ToxicTextCLIP: Text-Based Poisoning and Backdoor Attacks on CLIP Pre-training (NeurIPS 2025 · similarity 0.90)
- Defending Multimodal Backdoored Models by Repulsive Visual Prompt Tuning (NeurIPS 2025 · similarity 0.89)
- Test-Time Multimodal Backdoor Detection by Contrastive Prompting (ICML 2025 · similarity 0.88)
- Detecting Backdoor Samples in Contrastive Language Image Pretraining (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
