Research map / Adversarial robustness and security / Backdoor and poisoning

ICLShield: Exploring and Mitigating In-Context Learning Backdoor Attacks

ICML 2025 · Adversarial robustness and security › Backdoor and poisoning · approach: Backdoors in large multimodal models · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a dual-learning hypothesis and theoretical upper bound for in-context learning backdoor attacks, and ICLShield, a defense that adds clean demonstrations selected by similarity and confidence to raise the concept preference ratio.

Problem

In-context learning (ICL) in LLMs is vulnerable to backdoor attacks via poisoned demonstrations; prior work lacks a mechanistic explanation and no defense specifically targets ICL backdoors, since weight-level and training-time defenses do not apply.

Approach

A dual-learning hypothesis models the model as learning a task latent concept and a backdoor latent concept from poisoned demonstrations; the attack success upper bound is governed by their posterior ratio. ICLShield raises this ratio by appending extra clean demonstrations chosen by embedding similarity to the poisoned demonstration and by model confidence.

Key results

• On SST-2 with GPT-NEO-1.3B, ICLShield reduces ASR from 92.19% to 35.97% while ONION and Back-Translation only reach 98.13% and 82.30%; the paper reports +26.02% average improvement over baselines.
• On BadChain with LLaMA2-7B on GSM8K, ICLShield lowers ASR from 86.28% to 14.86%, versus about 85-86% for ONION and Back-Translation.
• On closed-source GPT-4o for SST-2, ICLShield reduces ASR from 7.16% to 3.88% using demonstrations selected on open-source models.

Measured effects reported in the paper

Surprising findings

• Simple inference-time defenses ONION and Back-Translation reduce ASR by only about 3.5% and 3.1% on average, showing they are ineffective against ICL backdoors.
• Randomly selected clean demonstrations also reduce attack success, supporting the claim that adding clean examples alone helps.

Limitations

• Theoretical assumptions (e.g., conditional probabilities equal to 1, independence of posteriors) are idealized and only partially validated by a single experiment on OPT-1.3B.
• Closed-source models use demonstrations selected on open-source models because output probabilities and embeddings are not accessible.
• Evaluated only on relatively small tasks and does not test complex prompting such as tree-of-thought.

Experimental setup

Datasets: SST-2 AG's News Stanford Alpaca AdvBench GSM8K CSQA

Benchmarks: SST-2 AG's News Sentiment steering (BackdoorLLM) Targeted refusal (BackdoorLLM) GSM8K CSQA

Metrics: clean accuracy attack success rate ASR with trigger ASR without trigger

Baselines: ONION Back-Translation No defense (ICLAttack)

Builds on: latent concept theory of in-context learning ICLAttack BadChain ONION Back-Translation

Closest papers by meaning

• BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language Models (NeurIPS 2025 · similarity 0.90)
• Simulate and Eliminate: Revoke Backdoors for Generative Large Language Models (AAAI 2025 · similarity 0.89)
• Backdoor Token Unlearning: Exposing and Defending Backdoors in Pretrained Language Models (AAAI 2025 · similarity 0.89)
• Revisiting Backdoor Attacks against Large Vision-Language Models from Domain Shift (CVPR 2025 · similarity 0.88)
• BadChain: Backdoor Chain-of-Thought Prompting for Large Language Models (ICLR 2024 · similarity 0.88)
• Causality Based Front-door Defense Against Backdoor Attack on Language Models (ICML 2024 · similarity 0.88)