Research map / Adversarial robustness and security / Backdoor and poisoning

Revisiting Backdoor Attacks against Large Vision-Language Models from Domain Shift

CVPR 2025 · Adversarial robustness and security › Backdoor and poisoning · approach: Backdoors in large multimodal models · 8 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Studies backdoor attacks on instruction-tuned LVLMs under train/test domain shift, introduces backdoor domain generalization (ASR-G) and proposes MABA, an attribution-guided multimodal trigger attack.

Problem

Backdoor attacks on LVLMs during instruction tuning are usually evaluated under matched train/test distributions, so their effectiveness under real-world domain shifts in visual and text inputs is unknown.

Approach

Defines backdoor domain generalization as a new evaluation dimension using a domain-shifted instruction dataset (diffusion-based image style changes and LLM-based text rewrites), derives two insights (domain-independent triggers and trigger-preferred prediction regions), and uses them to design MABA, which places a yellow oval visual trigger and text symbols at attribution-identified critical regi

Key results

• MABA raises mean ASR-G of BadNets from 0.318 to 0.682 (+36.4%) across OpenFlamingo, Otter and BLIP-2 under realism and expressionism shifts.
• MABA* extends DualKey with multimodal triggers, raising mean ASR-G from 0.638 to 0.834.
• All evaluated attacks reach up to 97% ASR at a 0.2% poisoning rate in the 350K-example realistic setting.

Measured effects reported in the paper

Surprising findings

• Blended, a simple static trigger, shows the strongest cross-domain generalization (mean ASR-G 0.986), while BadNets fails under image shift despite high in-domain ASR.
• Answer-domain shifts tend to increase rather than decrease attack generalizability.

Limitations

• Does not study how backdoor generalizability varies across different LVLM architectures.
• Does not examine defense mechanisms in depth.
• Evaluation focuses on image captioning with zero-shot classification and a limited set of victim models.

Experimental setup

Datasets: MIMIC-IT COCO Flickr30K M3IT CC3M

Benchmarks: COCO Flickr30K

Metrics: ASR ASR-G CIDEr ACC

Baselines: BadNets Blended WaNet LowFrequency InputAware DualKey TextBadNets AddSent StyleBkd GCG

Builds on: BadNets DualKey Blended CLIP-based attribution (Chen et al.)

Closest papers by meaning

• BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language Models (NeurIPS 2025 · similarity 0.90)
• Semantic Shield: Defending Vision-Language Models Against Backdooring and Poisoning via Fine-grained Knowledge Alignment (CVPR 2024 · similarity 0.90)
• BadVLA: Towards Backdoor Attacks on Vision-Language-Action Models via Objective-Decoupled Optimization (NeurIPS 2025 · similarity 0.90)
• CROW: Eliminating Backdoors from Large Language Models via Internal Consistency Regularization (ICML 2025 · similarity 0.89)
• Stealthy Backdoor Attack in Self-Supervised Learning Vision Encoders for Large Vision Language Models (CVPR 2025 · similarity 0.89)
• BadEdit: Backdooring Large Language Models by Model Editing (ICLR 2024 · similarity 0.89)