Research map / Large language models / Language, safety and interpretability
Adversarial Representation Engineering: A General Model Editing Framework for Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Adversarial Representation Engineering (ARE), which alternates training a representation discriminator and LoRA-editing an LLM so its hidden representations of anti-target prompts match target-concept representations, for jailbreak attack/defense and hallucination editing.
Problem
Model editing of LLM concepts (e.g., alignment, hallucination) via representation engineering is impractical: RepE vector injection can disrupt general performance and yields unreliable, overfit representation vectors.
Approach
Replace a fixed RepE contrast vector with a discriminator trained on the model's hidden representations, then adversarially fine-tune the LLM (via LoRA on selected layers) to fool the frozen discriminator, alternating updates in a GAN-like loop.
Key results
- ARE reduces refusal rate on AdvBench to 0.5% on Llama-2-7B-Chat, 0.0% on Vicuna-7B and Guanaco-7B, versus best baseline Contrast Vector at 5.9%, 1.1%, 0.9%.
- As a defense, ARE raises refusal under AutoDAN and GCG on Llama-2-7B-Chat to 99.0% and 89.7%, compared with 85.6%/60.2% for Self-Reminder.
- For discouraging hallucination on TruthfulQA, ARE reaches 52.14% right answers on Llama2-7B versus 36.84% for ITI and 34.27% for Self-Reminder.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ARE instead of Contrast Vector to remove alignment | Llama-2-7B-Chat · AdvBench attack | refusal rate: 5.9 -> 0.5 | Helped |
| Use ARE instead of Contrast Vector to remove alignment | Vicuna-7B · AdvBench attack | refusal rate: 1.1 -> 0.0 | Helped |
| Use ARE defense instead of ICD | Llama-2-7B-Chat · AdvBench, AutoDAN | refusal rate: 92.3 -> 99.0 | Helped |
| Use ARE defense instead of ICD | Llama-2-7B-Chat · AdvBench, GCG | refusal rate: 74.0 -> 89.7 | Helped |
| Use ARE defense instead of Self-Reminder | Vicuna-7B · AdvBench, AutoDAN | refusal rate: 12.3 -> 44.3 | Helped |
| Use ARE defense instead of ICD | Vicuna-7B · AdvBench, GCG | refusal rate: 18.4 -> 32.3 | Helped |
| Use ARE to encourage hallucinations instead of Self-Reminder | Llama2-7B · TruthfulQA | correct answer rate: 25.95 -> 11.75 | Hurt |
| Use ARE to encourage hallucinations instead of Self-Reminder | Mistral-7B · TruthfulQA | correct answer rate: 40.26 -> 22.03 | Hurt |
| Use ARE to discourage hallucinations instead of ITI | Llama2-7B · TruthfulQA | correct answer rate: 36.84 -> 52.14 | Helped |
| Use ARE to discourage hallucinations instead of ITI | Mistral-7B · TruthfulQA | correct answer rate: 45.17 -> 60.10 | Helped |
| Use ARE instead of Shadow Alignment | Llama2-7B · Text generation quality | Self-BLEU: 0.215 -> 0.078 | Helped |
| Use ARE instead of HEDA | Llama2-7B · Text generation quality | Repetition-4: 15.78 -> 7.53 | Helped |
Surprising findings
- Table 2 shows that ARE-defended Vicuna-7B has lower refusal under AutoDAN/GCG than some would expect, and ARE-edited Vicuna's refusal under no attack is 100%, but the attack-side results show ARE sometimes lower than baselines for the same model, suggesting inconsistent reporting.
Limitations
- Can be misused to bypass safety mechanisms and generate harmful content.
- Relies on curated human- or AI-crafted sequence datasets representing target concepts, which may limit generalizability.
- Evaluated only on a few 7B-scale models and two concepts (alignment and hallucination).
Experimental setup
Datasets: RepE dataset AdvBench Alpaca Instruction Tuning Dataset TruthfulQA
Benchmarks: AdvBench TruthfulQA
Metrics: refusal rate attack success rate % right answer Repetition-4 Repetition-Sen Self-BLEU
Baselines: In-Context Attack DeepInception GCG AutoDAN Contrast Vector Shadow Alignment HEDA Self-Reminder In-Context Defense Inference-Time Intervention
Builds on: Representation Engineering (RepE) GAN LoRA ReFT
Closest papers by meaning
- Attribution Analysis Meets Model Editing: Advancing Knowledge Correction in Vision Language Models with VisEdit (AAAI 2025 · similarity 0.87)
- Exposing the Illusion of Erasure in Knowledge Editing for LLMs (NeurIPS 2026 · similarity 0.87)
- Resolving Lexical Bias in Model Editing (ICML 2025 · similarity 0.87)
- Stealth edits to large language models (NeurIPS 2024 · similarity 0.86)
- VLKEB: A Large Vision-Language Model Knowledge Editing Benchmark (NeurIPS 2024 · similarity 0.86)
- Spectral Editing of Activations for Large Language Model Alignment (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.