AI Paper Atlas

Research map / Large language models / Instruction and fine-tuning

On the Exploitability of Instruction Tuning

Manli Shu, Jiongxiao Wang, Chen Zhu, Jonas Geiping, Chaowei Xiao, Tom Goldstein

NeurIPS 2023 · Large language models › Instruction and fine-tuning · approach: Instruction data selection · 13 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
generate poisoned responses with an oracle LLM instead of hand-crafted templates (over-refusal)AutoPoison · Poisoned training data, over-refusalperplexity: 8.32 -> 3.68Improved
generate poisoned responses with an oracle LLM instead of hand-crafted insertion (content injection)AutoPoison · Poisoned training data, content injectionperplexity: 7.38 -> 4.86Improved
oracle-generated poisons instead of one-phrase insertion into human responsesAutoPoison · Poisoned training data, content injectioncoherence: 0.58 -> 0.51Hurt
oracle-generated informative refusals instead of a fixed template replyAutoPoison · Poisoned training data, over-refusalcoherence: 0.04 -> 0.59Improved
AutoPoison instead of hand-crafted template for over-refusal poisoning of the modelOPT models fine-tuned on GPT-4-LLM · Dolly-15k outputs, over-refusalMAUVE: 0.38 -> 0.58 (OPT-350M, 10%)Improved
increase model size from smaller to larger OPTAutoPoison content injection · keyphrase occurrenceskeyphrase occurrence rateImproved
use the more recent Llama-2-7B instead of Llama-7B / OPT-6.7B as victimcontent injection attack · keyphrase occurrences at poison ratio <=5%keyphrase occurrence rateHurt
replace GPT-3.5-turbo oracle with Llama-2-chat-13B (ablation)AutoPoison content injection · keyphrase occurrenceskeyphrase occurrence rateNo clear effect
add persona system prompt 'cautious assistant' for the oracle (AutoPoison-PE) (ablation)AutoPoison over-refusal · informative refusalsinformative refusal countImproved
inject poisoned data at 10% ratio (attack) vs clean modelOPT-1.3B · TruthfulQAMC1/MC2: 0.252 -> 0.263 (MC1, injection)No clear effect
over-refusal AutoPoison at 10% poison vs clean modelOPT-1.3B · MMLU Security studiesaccuracy: 24.49 -> 20.82Hurt
AutoPoison poisoning at 10% vs clean modelOPT-1.3B · MMLU average over 57 tasksaccuracy: 25.39 -> 25.36No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy