Research map / Adversarial robustness and security / Attacks on text and graphs
HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces HarmBench, a standardized benchmark of 510 harmful behaviors and an evaluation pipeline for automated red teaming, runs 18 attacks against 33 LLMs/defenses, and proposes R2D2 adversarial training for robust refusal.
Problem
Automated red teaming methods are evaluated with disparate, non-comparable setups that lack breadth, standardized parameters, and robust classifiers, making it hard to rank attacks and defenses.
Approach
A standardized benchmark with diverse functional categories (standard, copyright, contextual, multimodal), fixed generation length (512 tokens), a validation/test split with held-out classifiers, and a fine-tuned Llama 2 13B judge distilled from GPT-4; plus R2D2, an efficient GCG-based adversarial training method using persistent test cases.
Key results
- Zephyr 7B + R2D2 reduces GCG attack success rate from 31.8% (Llama 2 7B Chat) and 30.2% (Llama 2 13B Chat) to 5.9%, the lowest among model-level defenses against GCG.
- The HarmBench fine-tuned classifier reaches 93.19% average agreement with human labels, above GPT-4 (88.37%) and other prior classifiers.
- Across 18 red teaming methods and 33 LLMs/defenses, no attack or defense is uniformly effective, and robustness shows no correlation with model size within model families.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add R2D2 adversarial training to Zephyr 7B | Zephyr 7B · GCG attacks | ASR (%): 31.8 -> 5.9 | Helped |
| Add R2D2 adversarial training to Zephyr 7B | Zephyr 7B · GCG attacks | ASR (%): 30.2 -> 5.9 | Helped |
| Add R2D2 adversarial training | Zephyr 7B · All evaluated attacks | ASR | Helped |
| Add R2D2 adversarial training | Zephyr 7B · MT-Bench | score: 6.0 vs. 6.5 | Mixed |
| Use the HarmBench classifier instead of GPT-4 | Classifier evaluation on the validation set · Average across standard and contextual behaviors | agreement rate: 88.37 -> 93.19 | Helped |
| Use the HarmBench classifier instead of GPT-4 | Classifier evaluation on the validation set · Standard behaviors | agreement rate: 89.8 -> 94.53 | Helped |
| Use the HarmBench classifier instead of GPT-4 | Classifier evaluation on the validation set · Contextual behaviors | agreement rate: 85.5 -> 90.5 | Helped |
| Use the HarmBench classifier instead of GPT-4₍PAIR₎ | Classifier prequalification evaluation · Three prequalification sets, average | accuracy: 89.6 -> 95.7 | Helped |
Surprising findings
- Robustness is largely independent of model size within a family, while differences between model families are large.
- Changing the generation token limit can shift substring-matching ASR by up to 30%.
Limitations
- Evaluation focuses on model-level defenses and text plus image inputs; system-level defenses are not evaluated.
- R2D2 is less effective against attacks dissimilar to the train-time GCG adversary, such as PAIR, TAP, and Stochastic Few-Shot.
Experimental setup
Datasets: AdvBench TDC 2023 Red Teaming Track UltraChat
Benchmarks: HarmBench MT-Bench
Metrics: attack success rate agreement with human labels
Baselines: GCG PAIR TAP AutoDAN PAP Llama 2 7B Chat Llama 2 13B Chat
Builds on: GCG Zephyr AdvBench PAIR TAP
Closest papers by meaning
- Learning Diverse Attacks on Large Language Models for Robust Red-Teaming and Safety Tuning (ICLR 2025 · similarity 0.90)
- Automated Red Teaming with GOAT: the Generative Offensive Agent Tester (ICML 2025 · similarity 0.89)
- There are Levels to It: Red Teaming LLMs with Hierarchical Reinforcement Learning (NeurIPS 2026 · similarity 0.89)
- AutoRedTeamer: Autonomous Red Teaming with Lifelong Attack Integration (NeurIPS 2025 · similarity 0.89)
- WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Models (NeurIPS 2024 · similarity 0.89)
- ReaLM: A Unified Red-Teaming Benchmark for Physical-World VLMs (NeurIPS 2026 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.