Research map / Adversarial robustness and security / Backdoor and poisoning
Virus Infection Attack on LLMs: Your Poisoning Can Spread "VIA" Synthetic Data
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Shows that poisoning and backdoor attacks rarely propagate through LLM synthetic data, and proposes Virus Infection Attack (VIA), which hides payloads in benign SFT responses at high-frequency hijacking points wrapped in a LLM-generated shell so they spread into synthetic data.
Problem
Current data poisoning and backdoor attacks on LLMs rarely appear in synthetic data generated from a poisoned upstream model, so it is unclear whether unsafe content can propagate to downstream models through synthetic training data.
Approach
Embed the poisoning payload inside benign training responses at a frequent, low-predictability hijacking anchor (found by an n-gram score), wrapped in a fluent LLM-generated shell, so the poisoned upstream model emits the payload even for clean, unrelated queries and propagates it into synthetic data.
Key results
- Vanilla poisoning yields near-zero infection rate in synthetic data (below 0.1%), e.g., 0.00 IR for sentiment steering, while poisoned upstream ASR is 36-100%.
- VIA with HPS (4-gram) raises the synthetic-data IR for sentiment steering to 85.64% and knowledge injection to 62.38%, versus 0.20% for CoT/response poisoning.
- VIA (mixup) keeps high upstream ASR (e.g., 100% on NegSentiment for BadNet) while raising downstream IR to about 70-79%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use VIA to embed poisoned payloads in benign samples | Data poisoning attacks · Sentiment Steering and Knowledge Injection | IR: 0.20 -> 72.44; 0.22 -> 28.68 | Helped |
| Use VIA with HPS (4-gram) instead of CoT/Response Poisoning | Data poisoning attacks · Sentiment Steering and Biased Recommendation | ASR: 100.00 -> 53.65; 5.26 -> 68.42 | Mixed |
| Add VIA to existing backdoor methods | Backdoor attacks · Jailbreaking, Negative Sentiment, and Refusal | IR: BadNet: 0.05 -> 64.53; 0.15 -> 52.97; 0.02 -> 56.92 | Helped |
| Use VIA (mixup) with BadNet | BadNet · Negative Sentiment and Refusal | ASR: 56.57 -> 100.00; 58.29 -> 100.00 | Helped |
| Use VIA (mixup) with BadNet | BadNet · Negative Sentiment and Refusal | IR: 52.97 -> 70.82; 56.92 -> 78.72 | Helped |
| Use VIA with HPS (4-gram) instead of HPS (3-gram) (ablation) | VIA data poisoning · Sentiment Steering, Knowledge Injection, and Biased Recommendation | IR: 72.44 -> 85.64; 28.68 -> 62.38; 66.14 -> 87.82 | Helped |
| Move the payload injection position to the end instead of using HPS (3-gram) (ablation) | VIA data poisoning · Sentiment Steering and Biased Recommendation | IR: 72.44 -> 77.96; 66.14 -> 73.38 | Helped |
| Use random payload injection instead of HPS (3-gram) (ablation) | VIA data poisoning · Knowledge Injection | IR: 28.68 -> 40.38 | Helped |
| Add semantic similarity search (SS) to HPS (3-gram) (ablation) | VIA data poisoning · Knowledge Injection | IR: 28.68 -> 57.48 | Helped |
| Add semantic similarity search (SS) to HPS (3-gram) (ablation) | VIA data poisoning · Sentiment Steering and Biased Recommendation | IR: 72.44 -> 57.92; 66.14 -> 58.00 | Hurt |
| Use LLM-based rather than fixed-format shell construction (ablation) | VIA with SS · Sentiment Steering | IR: 57.92 -> 22.98 | Hurt |
| Use LLM-based rather than fixed-format shell construction (ablation) | VIA with SS · Sentiment Steering | ASR: 46.34 -> 78.04 | Helped |
Surprising findings
- Poisoned upstream models rarely emit their own poisoning content in synthetic data, so synthetic-data training is resilient to existing attacks.
- Increasing the upstream poisoning rate to 40% does not increase poisoning in synthetic data, because the adversary cannot control the query distribution.
Limitations
- VIA supports only text/language models; extension to other modalities is not addressed.
- VIA lowers upstream ASR to about 40-70% unless mixed with vanilla poisoning, and the proposed defenses are ineffective against SC-enhanced VIA.
- Greedy decoupled optimization of hijacking point and wrapper may not be globally optimal.
Experimental setup
Datasets: Tulu-3 OpenO1-SFT Alpaca HH-RLHF
Metrics: attack success rate (ASR) infection rate (IR) recall false positive rate F1 score
Baselines: Unsupervised Text Poisoning CoT/Response Poisoning BadNet CTBA MTBA VPI Sleeper Agent
Builds on: BackdoorLLM DetectGPT
Closest papers by meaning
- Poisoning Attacks on LLMs Require a Near-constant Number of Poison Samples (NeurIPS 2026 · similarity 0.89)
- Persistent Pre-training Poisoning of LLMs (ICLR 2025 · similarity 0.87)
- Scaling Trends for Data Poisoning in LLMs (AAAI 2025 · similarity 0.86)
- The Canary’s Echo: Auditing Privacy Risks of LLM-Generated Synthetic Text (ICML 2025 · similarity 0.86)
- Expected Harm: Rethinking Safety Evaluation of (Mis)Aligned LLMs (NeurIPS 2026 · similarity 0.85)
- Demystifying Poisoning Backdoor Attacks from a Statistical Perspective (ICLR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.