AI Paper Atlas

Research map / Large language models / Instruction and fine-tuning

Be Careful When Fine-tuning On Open-Source LLMs: Your Fine-tuning Data Could Be Secretly Stolen!

Zhexin Zhang, Yuhao Sun, Junxiao Yang, Shiyao Cui, yuanchao zhang, Hongning Wang, Minlie Huang

ICLR 2026 · Large language models › Instruction and fine-tuning · approach: Safety alignment after fine-tuning · 3 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add SFT backdoor training with extraction instructionQwen2.5-7B · Dolly extractionmean match ratio (%): 13.4 -> 29.8Improved
use GRPO instead of SFT for backdoor trainingQwen2.5-7B · Dolly extractionmean match ratio (%): 29.8 -> 33.2Improved
use GRPO instead of SFT for backdoor trainingLlama3.1-8B · Dolly extractionmean match ratio (%): 43.3 -> 38.5Hurt
increase model size from 7B to 32BQwen2.5 SFT backdoor · Dolly extractionmean match ratio (%): 29.8 -> 49.2Improved
add backdoor trainingQwen2.5-7B · AlpacaEval 2 / MMLUwin rate / accuracy: 28.0 -> 33.0; MMLU 71.3 -> 71.3No clear effect
test with paraphrased extraction instruction instead of training instruction (ablation)SFT backdoor · Dolly extractionmean match ratio (%): 29.8 -> 17.6Hurt
train backdoor with fabricated trigger instruction (ablation)SFT backdoor · Dolly extraction detection by paraphrase probingmean match ratio (%): paraphrase-probe 17.6 -> 11.2Improved
add DP-SGD during downstream fine-tuning (ablation)Llama3.1-8B SFT backdoor · MATH extractionmean match ratio (%): 50.9 -> 0.9 (eps=4.0)Hurt
add DP-SGD during downstream fine-tuning (ablation)Llama3.1-8B · MATH500accuracy (%): 14.0 -> 1.2 (eps=4.0)Hurt
increase downstream fine-tuning epochs (ablation)Qwen2.5-7B SFT backdoor · Dolly extractionmatch ratioImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy