Research map / Adversarial robustness and security / Attacks on text and graphs
A Coin Flip for Safety: LLM Judges Fail to Reliably Measure Adversarial Robustness
Leo Schwinn, Moritz Ladenburger, Tim Beyer, Mehrnaz Mofakhami, Gauthier Gidel, Stephan Günnemann
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| restrict evaluation to 41 most consistent behaviors (ReliableBench) | LLM safety judges · HarmBench subset | average judge accuracy: 53% -> 70% | Improved |
| correct ASR by judge precision | attack success rate reporting · Gemma-3-1B | expected ASR: BoN falls to lowest corrected ASR | Improved |
| collect multiple judge-positive samples per behavior | attack evaluation · Gemma-3-1B | P(at least one true positive) | Improved |
| average scores over multiple judges (ensemble) (ablation) | LLM safety judges · Llama-3.1-8B | correlation with human rating (R2): 0.02 -> 0.05 | No clear effect |
| adjust classification threshold of judge (ablation) | JailJudge · Llama-3.1-8B | AUROC: 0.48 to 0.64 | No clear effect |
| use high inter-judge agreement as proxy for correctness | Judge Concordance score · HarmBench behaviors | judge accuracy | No clear effect |
| continue attack optimization (more GCG/GCG-R/PAIR steps) | adversarial attacks · HarmBench subset | human-rated harmfulness | No clear effect |
| switch victim model from Llama-3.1-8B to Gemma-27B | HarmBench judge on direct prompting · HarmBench subset | judge accuracy: 45% -> 58% | Improved |
| use judge-feedback in attack (GCG-REINFORCE) | attack · HarmBench subset | JailJudge AUROC: 0.64 (GCG) -> 0.48 | Hurt |
| evaluate judges on samples where all other judges fail | LlamaGuard-3 · JudgeStressTest | accuracy: 0.54 -> 0.17 | Hurt |
Closest papers by meaning
- On the Robustness of Verbal Confidence of LLMs in Adversarial Attacks (NeurIPS 2025 · similarity 0.90)
- Safeguarding LLMs via Model-Agnostic Latent Safety Signals from Dark Knowledge (NeurIPS 2026 · similarity 0.88)
- An LLM can Fool Itself: A Prompt-Based Adversarial Attack (ICLR 2024 · similarity 0.88)
- Navigating the Safety Landscape: Measuring Risks in Finetuning Large Language Models (NeurIPS 2024 · similarity 0.88)
- Safety Alignment Can Be Not Superficial With Explicit Safety Signals (ICML 2025 · similarity 0.88)
- JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
