Research map / Adversarial robustness and security / Attacks on text and graphs
Comparison requires valid measurement: Rethinking attack success rate comparisons in AI red teaming
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Position paper arguing that attack success rate (ASR) comparisons in AI red teaming often lack conceptual coherence and measurement validity, illustrated with jailbreaking case studies and measurement-theory framing.
Problem
ASR comparisons across systems and jailbreak methods are routinely used to draw conclusions about relative safety or attack efficacy, but these comparisons often rest on incongruent estimands or invalid measurements.
Approach
Frames ASRs as estimates of estimands defined by a probabilistic threat model and applies social-science measurement theory (systematization, operationalization, validity) to give a two-part sufficient condition for meaningful ASR comparison: conceptual coherence and measurement validity.
Key results
- Using 49 decoding configurations on Llama 2 7B/13B, one-shot ASR stays around 0.2 while Top-1 ASR rises with temperature, showing the GE gain comes from Top-1 aggregation over repeated samples.
- Top-1 ASR over 50 repeated samples of baseline prompts on Llama 2 7B Chat at temperature 2.0 reaches 0.83, close to the reported best jailbreak LAA at 0.88.
- Of 10 'Political Activities' prompts used in a prior jailbreak study, only 1 is a clear policy violation, 7 are borderline, and 2 are clearly non-violating.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Top-1 aggregation over 392 sampled responses rather than one-shot evaluation | Generation Exploitation compared with GCG · Llama 2 13B-Chat | ASR: 0.31 -> 0.89 | Mixed |
| Use high-entropy decoding configurations with Top-1 aggregation over repeated samples | Baseline prompting · MaliciousInstruct, 100 prompts | Top-1 ASR | Helped |
| Vary decoding configuration while measuring one-shot ASR | Llama 2 7B and 13B chat · MaliciousInstruct, 100 prompts | one-shot ASR | No clear effect |
| Repeatedly sample unmodified prompts 50 times at temperature 2.0 | Llama 2 7B Chat · 160 base prompts | Top-1 ASR: 0.83 | Helped |
| Use the attacker scorer instead of the judge to select the response | Generation Exploitation · MaliciousInstruct | ASR | Hurt |
| Use prompts that do not clearly represent policy-violating harmful requests | Baseline safety evaluation · Political Activities category | ASR validity | Hurt |
Surprising findings
- Repeated resampling of base prompts at high temperature can match sophisticated jailbreak methods on Top-1 ASR.
- Equal overall judge accuracy across systems does not prevent ASR bias; differential TPR/FPR can make equally safe systems appear different.
Limitations
- Analysis is largely idealized, assuming perfect operationalization in the conceptual coherence section.
- Judge accuracy numbers from prior studies are hard to interpret without consistent systematized success criteria.
- Framework is less suited to qualitative red teaming.
Experimental setup
Datasets: MaliciousInstruct HarmBench-style 160 base prompts (JailbreakRadar)
Benchmarks: MaliciousInstruct JailbreakRadar 160 prompts
Metrics: attack success rate Top-1 ASR one-shot ASR TPR FPR
Baselines: GCG Generation Exploitation GPTFuzzer Best-of-N jailbreak LAA
Builds on: GCG Generation Exploitation HarmBench GPTFuzzer JailbreakBench
Closest papers by meaning
- Security Challenges in AI Agent Deployment: Insights from a Large Scale Public Competition (NeurIPS 2025 · similarity 0.88)
- Attack Selection In Agentic AI Control Evaluations Meaningfully Decreases Safety (NeurIPS 2026 · similarity 0.86)
- HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal (ICML 2024 · similarity 0.85)
- ReaLM: A Unified Red-Teaming Benchmark for Physical-World VLMs (NeurIPS 2026 · similarity 0.84)
- Position: AI Safety Must Embrace an Antifragile Perspective (ICML 2025 · similarity 0.84)
- AutoRedTeamer: Autonomous Red Teaming with Lifelong Attack Integration (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.