Research map / Large language models / Instruction and fine-tuning
Poisoning Language Models During Instruction Tuning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add dirty-label poison examples | Tk-Instruct · Held-out polarity datasets with trigger | misclassification rate: 92.8% | Helped |
| Add correctly labeled poison examples | Tk-Instruct · Held-out polarity datasets with trigger | misclassification rate: 55.6% | Helped |
| Increase clean-label poison examples from 100 to 200 and then 400 (ablation) | Tk-Instruct · Held-out polarity datasets with trigger | misclassification rate: 55.6% -> 71.6% -> 77.6% | Helped |
| Use random rather than bag-of-n-grams-selected poison examples (ablation) | Tk-Instruct · Held-out sentiment datasets | misclassification rate: 69.2% vs. 83.8% | Helped |
| Use bag-of-n-grams-selected rather than random poison examples (ablation) | Tk-Instruct · Held-out toxicity datasets | misclassification rate: 70.7% vs 37.7% | Helped |
| Set poison outputs to repeat the trigger phrase rather than random tokens (ablation) | Tk-Instruct arbitrary-task poisoning · Held-out arbitrary tasks with trigger | average accuracy drop: 20.0 -> 39.3 | Helped |
| Distribute a fixed number of poison samples across more tasks (ablation) | Tk-Instruct arbitrary-task poisoning · Held-out arbitrary tasks with trigger | average accuracy drop: 10.6 vs 8.1 | Helped |
| Reduce poison examples to five per task (ablation) | Tk-Instruct arbitrary-task poisoning · Held-out arbitrary tasks with trigger | average accuracy drop: 38.8 | Helped |
| Use an 11-billion parameter model instead of smaller models (ablation) | Tk-Instruct arbitrary-task poisoning · Held-out arbitrary tasks with trigger | average accuracy drop: 36.9 vs 25.0 | Mixed |
| Stop training after two epochs rather than ten (ablation) | Tk-Instruct 3B polarity poisoning · Polarity validation set with trigger | misclassification rate: 92.8% -> 21.4% | Hurt |
| Lower the learning rate from 1e-5 to 1e-6 (ablation) | Tk-Instruct 3B polarity poisoning · Polarity validation set with trigger | misclassification rate: 29.9% | Hurt |
| Filter the top 6.3% highest-loss samples and retrain | Tk-Instruct 3B polarity poisoning · Polarity validation set with trigger | misclassification rate: 35.2% | Hurt |
Closest papers by meaning
- Poisoning Attacks on LLMs Require a Near-constant Number of Poison Samples (NeurIPS 2026 · similarity 0.90)
- PoisonBench: Assessing Language Model Vulnerability to Poisoned Preference Data (ICML 2025 · similarity 0.88)
- Persistent Pre-training Poisoning of LLMs (ICLR 2025 · similarity 0.88)
- BadEdit: Backdooring Large Language Models by Model Editing (ICLR 2024 · similarity 0.87)
- Antidote: Post-fine-tuning Safety Alignment for Large Language Models against Harmful Fine-tuning Attack (ICML 2025 · similarity 0.86)
- Do as I do (Safely): Mitigating Task-Specific Fine-tuning Risks in Large Language Models (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.