Research map / Large language models / Instruction and fine-tuning

Simulate and Eliminate: Revoke Backdoors for Generative Large Language Models

AAAI 2025 · Large language models › Instruction and fine-tuning · approach: Safety alignment after fine-tuning · 5 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SANDE, which removes backdoors from generative LLMs without a clean reference model by overwriting backdoor mappings via OSFT, with a parrot-prompt simulation stage when triggers are unknown.

Problem

Backdoors implanted in generative LLMs during pre-training or fine-tuning survive standard safety training (SFT, RLHF), and existing defenses focus on detection or require a clean reference model. The paper targets removing such backdoors when the trigger and/or triggered response is unknown.

Approach

Overwrite SFT (OSFT) maps trigger-containing prompts to clean golden responses, overwriting the trigger-to-response mapping. For unknown triggers, a learnable soft "parrot prompt" is tuned to reproduce the triggered response (simulation), then OSFT is applied with the parrot prompt prepended (elimination). No clean reference model is required.

Key results

• SANDE and OSFT reduce ASR to near 0% (e.g., Llama2-Alpaca 0.0 for SANDE, 0.02 for OSFT) while Fine-mixing also reaches 0% but requires a clean pre-trained model; SFT and DPO leave ASR near 100% in-domain.
• OSFT is the least harmful to utility, e.g., Llama2-Alpaca MMLU 39.29 vs baseline 40.20, while SANDE-P and SANDE lose more accuracy (e.g., SANDE MMLU 38.06 on Llama2-Alpaca).
• Out-of-domain removal with SFT is effective on Llama2 (ASR 5.66 and 1.15) but not on Qwen1.5 (ASR 98.59 and 99.05), contrary to in-domain findings.

Measured effects reported in the paper

Surprising findings

• Standard SFT and DPO fail to remove the backdoor in-domain (ASR stays above 94% in most settings), while SFT with out-of-domain data sometimes removes it on Llama2.
• Fine-mixing, which needs a clean pre-trained model, is the only baseline to reach 0% ASR in-domain on all four setups, matching the proposed methods.

Limitations

• SANDE and SANDE-P incur larger utility degradation than OSFT, indicating learned parrot prompts are less faithful than the true trigger.
• Experiments cover only two models (Llama2-7b, Qwen1.5-4b), a single trigger phrase in the main results, and a fixed-length, first-token partial response for SANDE-P.
• Parrot prompt can simulate only one trigger at a time, as acknowledged in future work.

Experimental setup

Datasets: Stanford Alpaca OpenOrca Anthropic HH-RLHF

Benchmarks: MMLU ARC-e ARC-c

Metrics: ASR clean accuracy

Baselines: SFT DPO NAD Fine-mixing Baseline (backdoored model)

Builds on: prompt tuning Fine-mixing NAD DPO

Closest papers by meaning

• BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language Models (NeurIPS 2025 · similarity 0.94)
• BadEdit: Backdooring Large Language Models by Model Editing (ICLR 2024 · similarity 0.91)
• Backdoor Token Unlearning: Exposing and Defending Backdoors in Pretrained Language Models (AAAI 2025 · similarity 0.90)
• Probe before You Talk: Towards Black-box Defense against Backdoor Unalignment for Large Language Models (ICLR 2025 · similarity 0.90)
• Watch Out for Your Guidance on Generation! Exploring Conditional Backdoor Attacks against Large Language Models (AAAI 2025 · similarity 0.90)
• BadChain: Backdoor Chain-of-Thought Prompting for Large Language Models (ICLR 2024 · similarity 0.90)