Research map / Adversarial robustness and security / Backdoor and poisoning

Poisoning Attacks on LLMs Require a Near-constant Number of Poison Samples

NeurIPS 2026 · Adversarial robustness and security › Backdoor and poisoning · approach: Backdoors in large multimodal models · 7 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that backdoor data-poisoning success in LLM pretraining (600M–13B, Chinchilla-optimal) and fine-tuning depends on a near-constant absolute number of poisoned documents (~250) rather than poisoning percentage.

Problem

Existing pretraining poisoning studies assume a fixed percentage of training data is poisoned, which implies impractically large poison volumes for large models; it is unclear how many poisoned samples are needed as model and dataset size scale.

Approach

Hold the number of poisoned documents fixed while scaling model and clean-data size (Chinchilla-optimal), showing attack success tracks absolute poison count rather than poisoning rate, in both pretraining and safety fine-tuning.

Key results

• Across 600M–13B pretrained models, 250 poisoned documents reliably backdoor models (perplexity increase above 200 by end of training) despite up to 20x more clean data for the largest model.
• 100 poisoned documents were not successful in pretraining, while 250 and 500 were.
• In Llama-3.1-8B-Instruct fine-tuning, the absolute poison count, not poison ratio, dominates ASR even when clean data grows by two orders of magnitude (1k to 100k).

Surprising findings

• Continued clean pretraining degrades language-switch backdoors only slowly, and poisoning at the beginning vs end of fine-tuning yields very different poison-count requirements.
• Poisoning at the start of fine-tuning with 20 samples works but the same 20 samples at the end does not, suggesting path dependence.

Limitations

• Backdoors were not shown to persist through realistic safety post-training, so end-to-end attacks are not demonstrated.
• Only a narrow set of backdoor behaviours (denial-of-service, language switch, harmful compliance) and triggers were studied.
• Pretraining experiments limited to dense models up to 13B; mitigations and defences not explored.

Experimental setup

Datasets: Pile Pile validation StrongReject Alpaca-style non-harmful instruction data

Benchmarks: ARC Lambada OpenAI LogiQA PIQA SciQ WinoGrande Hellaswag

Metrics: perplexity increase per token attack success rate clean accuracy near-trigger accuracy StrongReject-style compliance rate

Baselines: Pythia-6.9B

Builds on: Chinchilla scaling laws Pythia StrongReject

Closest papers by meaning

• Persistent Pre-training Poisoning of LLMs (ICLR 2025 · similarity 0.93)
• PoisonBench: Assessing Language Model Vulnerability to Poisoned Preference Data (ICML 2025 · similarity 0.92)
• Scaling Trends for Data Poisoning in LLMs (AAAI 2025 · similarity 0.91)
• Virus Infection Attack on LLMs: Your Poisoning Can Spread "VIA" Synthetic Data (NeurIPS 2025 · similarity 0.89)
• BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language Models (NeurIPS 2025 · similarity 0.89)
• Forget to Flourish: Leveraging Machine-Unlearning on Pretrained Language Models for Privacy Leakage (AAAI 2025 · similarity 0.89)