Research map / Adversarial robustness and security / Backdoor and poisoning
LT-Defense: Searching-free Backdoor Defense via Exploiting the Long-tailed Effect
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LT-Defense, a searching-free backdoor detection method for language models that exploits the long-tailed effect of poisoned data using clean examples and two metrics (HFR and ATS), plus test-time backdoor freezing and attack target prediction.
Problem
Backdoor detection in language models via trigger or target search is computationally expensive, especially when the output space is large (e.g., full vocabulary for token prediction), and existing methods are costly across many candidate targets.
Approach
Poisoned data makes the target class a head class, creating a long-tailed effect that shifts benign-example activations toward attack targets; this can be measured with a few clean examples (Head-Feature Rate for task-agnostic, Abnormal Token Score for task-related), avoiding any trigger or target search.
Key results
- Task-agnostic backdoor detection reaches 98% average detection accuracy across 1440 models (720 benign, 720 poisoned), with time cost under 1% of LMSanitator.
- Test-time backdoor freezing reduces attack success rate to less than 1% in most cases on AG News with only microsecond-level overhead.
- Attack target prediction achieves 100% precision for single-token BToP and AutoPoison refusal attacks.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LT-Defense instead of LMSanitator for task-agnostic backdoor detection | RoBERTa-base · Task-agnostic detection, POR | ACC: 93.3 -> 100.0 | Helped |
| Use LT-Defense instead of LMSanitator for task-agnostic backdoor detection | RoBERTa-base · Task-agnostic detection, POR | Time: 180.0s -> 0.9s | Helped |
| Use LT-Defense for task-related backdoor detection | RoBERTa-large · PoisonPrompt on SST-2 | ACC: 0.92 | Mixed |
| Use LT-Defense to predict attack targets | RoBERTa-large · BToP | AMR: 1.0 | Helped |
| Use LT-Defense to predict attack targets | RoBERTa-large · PoisonPrompt | AMR: 0.68 | Hurt |
| Increase the number of clean test examples (ablation) | LT-Defense · Task-agnostic detection | Detection accuracy | Helped |
| Use WikiText or RTE as the test dataset (ablation) | LT-Defense · Task-agnostic detection | Detection accuracy | No clear effect |
| Vary the number or type of PVs (ablation) | LT-Defense · Task-agnostic detection | Detection precision | No clear effect |
| Increase the number of clean test examples (ablation) | LT-Defense · Task-related detection | Detection accuracy | Helped |
| Reduce poisoned features of PVs in an adaptive attack (ablation) | Task-agnostic backdoor attack against LT-Defense · Adaptive attack evaluation | Attack success rate | Hurt |
| Increase variance of clean feature activations in an adaptive attack (ablation) | Task-agnostic backdoor attack against LT-Defense · Adaptive attack evaluation | Attack success rate | Hurt |
| Reduce target-token logits on clean examples in an adaptive attack (ablation) | Task-related backdoor attack against LT-Defense · Adaptive attack evaluation | Attack success rate | Hurt |
Surprising findings
- Detection of PoisonPrompt attacks yields more false positives because the downstream task itself is imbalanced and induces a natural long-tailed effect.
Limitations
- Performance drops for few-target, imbalanced downstream tasks such as PoisonPrompt, producing false positives.
- Adaptive attacks were evaluated only under strong attacker assumptions; the defense's robustness to stronger adaptive attacks is not established.
- Hyperparameter thresholds (ts1, ts2, ts3) are set using reference benign models fine-tuned per architecture.
Experimental setup
Datasets: WikiText BookCorpus SST-2 AG News GPT-4-LLM Databricks-Dolly-15k RTE
Benchmarks: WikiText BookCorpus SST-2 AG News GPT-4-LLM Databricks-Dolly-15k
Metrics: false positive false negative detection accuracy average time benign accuracy attack success rate average token mapping rate
Baselines: LMSanitator ONION
Builds on: LMSanitator ONION P-Tuning v2
Closest papers by meaning
- Probe before You Talk: Towards Black-box Defense against Backdoor Unalignment for Large Language Models (ICLR 2025 · similarity 0.89)
- Causality Based Front-door Defense Against Backdoor Attack on Language Models (ICML 2024 · similarity 0.89)
- Backdoor Token Unlearning: Exposing and Defending Backdoors in Pretrained Language Models (AAAI 2025 · similarity 0.87)
- Demystifying Poisoning Backdoor Attacks from a Statistical Perspective (ICLR 2024 · similarity 0.87)
- BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language Models (NeurIPS 2025 · similarity 0.87)
- ICLShield: Exploring and Mitigating In-Context Learning Backdoor Attacks (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.