Research map / Adversarial robustness and security / Attacks on text and graphs
VL-Trojan: Multimodal Instruction Backdoor Attacks against Autoregressive Visual Language Models
Jiawei Liang, Siyuan Liang, Aishan Liu, Xiaochun Cao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use VL-Trojan multimodal trigger vs Badnet (in-task) | OpenFlamingo-3B · COCO | ASR (%): 37.30 -> 99.82 | Improved |
| use VL-Trojan trigger vs Badnet (spot-the-difference instruction data) | OpenFlamingo-3B · COCO | ASR (%): 26.80 -> 99.98 | Improved |
| add L2 (cluster poisoned features) to L1 image trigger (ablation) | VL-Trojan image trigger · COCO | ASR (%): 88.96 -> 96.68 | Improved |
| add L1 (separate poisoned from clean features) to L2 (ablation) | VL-Trojan image trigger · COCO | ASR (%): 16.64 -> 96.68 | Improved |
| add text trigger to image trigger at 0.25% poisoning (ablation) | VL-Trojan · COCO | ASR (%): 96.68 -> 97.86 | Improved |
| text trigger alone at 0.25% poisoning (ablation) | VL-Trojan text trigger · COCO | ASR (%): 0.96 | No clear effect |
| add text trigger with surrogate ViT-B/16 (black-box) (ablation) | VL-Trojan · COCO | ASR (%): 1.10 -> 97.72 | Improved |
| add text trigger with surrogate ViT-H/14 (black-box) (ablation) | VL-Trojan · COCO | ASR (%): 0.68 -> 99.64 | Improved |
| add text trigger with surrogate RN50 (black-box) (ablation) | VL-Trojan · COCO | ASR (%): 64.62 -> 99.78 | Improved |
| increase poisoning rate (ablation) | VL-Trojan · COCO | clean accuracy | Hurt |
| lower poisoning rate to 0.1% (23 samples) (ablation) | VL-Trojan · COCO | ASR (%): exceeds 80% | No clear effect |
| add more clean in-context examples (0 to 4 shot) | conventional backdoors · COCO | ASR (%) | Hurt |
Later papers that compared with this method
3 later accepted papers report VL-Trojan as a baseline in their results tables; 3 of them report a better number on most of the results they share with it.
- MTAttack: Multi-Target Backdoor Attacks Against Large Vision-Language Models (AAAI 2026 · better on 14 of 14 shared results)
- VLMSysTrojan: Stealthy System-Aware Backdoor Attacks Against Vision-Language Models (ECCV 2026 · better on 3 of 3 shared results)
- TokenSwap: Backdoor Attack on the Compositional Understanding of Large Vision-Language Models (ICML 2026 · better on 1 of 1 shared result)
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Revisiting Backdoor Attacks against Large Vision-Language Models from Domain Shift (CVPR 2025 · similarity 0.90)
- Backdooring Vision-Language Models with Out-Of-Distribution Data (ICLR 2025 · similarity 0.88)
- BadCLIP: Dual-Embedding Guided Backdoor Attack on Multimodal Contrastive Learning (CVPR 2024 · similarity 0.87)
- Token by Token, Compromised: Backdoor Vulnerabilities in Unified Autoregressive Models (NeurIPS 2026 · similarity 0.87)
- Defending Multimodal Backdoored Models by Repulsive Visual Prompt Tuning (NeurIPS 2025 · similarity 0.87)
- FusionAudit: Pathway-Conditioned Robustness Auditing for Native Multimodal Models (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
