Research map / Adversarial robustness and security / Black-box and transfer attacks

AnchorRep: Defending LLMs Against Cross-Model Adversarial Transfer via Representation Repulsion

NeurIPS 2026 · Adversarial robustness and security › Black-box and transfer attacks · approach: Transfer attacks on face recognition · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes AnchorRep, a LoRA adapter trained to repel a defended LLM's harmful-prompt representations from a frozen anchor model's (via CKA) with auxiliary utility-preserving losses, cutting cross-model GCG transfer ASR to about 1% and introducing the Benign Garble Rate metric.

Problem

Jailbreak attacks optimized on one open-weight LLM transfer to architecturally different models, and existing defenses (designed for same-model attacks) either fail to block transfer or degrade benign utility through garbled output or over-refusal.

Approach

Minimize linear CKA between the defended model's harmful-prompt hidden states and those of a frozen anchor model at a mid-depth layer via a LoRA adapter, combined with refusal-direction, coherency, and KL losses that keep benign behavior stable; no adversarial examples are used in training.

Key results

• AnchorRep reduces GCG transfer ASR on Mistral from 36.6% to 1.1% and on Llama-3 to 1.1% over 2,000 transferred attacks, with 0% benign garble rate.
• Circuit Breakers yields 77.1% BGR on Llama-3 with -48.6 OR-Bench delta, showing garbling invisible to refusal metrics.
• In white-box adaptive attacks, the defense reduces ASR in 22 of 25 attack-target settings, but Embedding PGD keeps 40-88% ASR on 14B models.

Surprising findings

• Existing defenses can reach low transfer ASR only by inducing large benign garbling (e.g., Circuit Breakers 77% BGR), which refusal-based metrics miss.
• Embedding-space PGD attacks retain high ASR (40-88%) on the 14B models despite the defense.

Limitations

• Embedding-space attacks (Embedding PGD) retain high ASR on 14B models
• TAP semantic-reframing partially bypasses the defense on Llama-3
• Several results rely on a single anchor choice and a per-model tuned configuration

Experimental setup

Datasets: AdvBench WikiText-2 XSTest GCG transfer suffixes (20 sources x 100 prompts) HarmBench OR-Bench Hard FalseReject MMLU MT-Bench

Benchmarks: GCG transfer (2,000 prompts) HarmBench OR-Bench Hard XSTest FalseReject MMLU MT-Bench

Metrics: attack success rate benign garble rate over-refusal delta MMLU MT-Bench

Baselines: Circuit Breakers RepBend RMU CRL SmoothLLM

Builds on: Circuit Breakers GCG CKA TRADES Platonic Representation Hypothesis

Closest papers by meaning

• PubDef: Defending Against Transfer Attacks From Public Models (ICLR 2024 · similarity 0.88)
• Anchoring Adversarial Trajectories to Data Manifolds: A Bilevel Transfer Optimization Framework (NeurIPS 2026 · similarity 0.87)
• Transferability Bound Theory: Exploring Relationship between Adversarial Transferability and Flatness (NeurIPS 2024 · similarity 0.86)
• Soft Prompt Threats: Attacking Safety Alignment and Unlearning in Open-Source LLMs through the Embedding Space (NeurIPS 2024 · similarity 0.85)
• LRS: Enhancing Adversarial Transferability through Lipschitz Regularized Surrogate (AAAI 2024 · similarity 0.85)
• Enhancing Adversarial Transferability with Adversarial Weight Tuning (AAAI 2025 · similarity 0.85)