Research map / Large language models / Language, safety and interpretability
SEAS: Self-Evolving Adversarial Safety Optimization for Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SEAS, a self-evolving red-teaming framework where a Red Team LLM and a Target LLM iteratively improve via DPO on adversarial prompts and safe/unsafe responses judged by Llama Guard 2, using a new 18K-entry safety dataset.
Problem
Automated red teaming struggles to generate diverse, complex adversarial prompts and to adapt to evolving, subtle vulnerabilities of LLM targets, so safety improvements rely heavily on costly manual red teaming.
Approach
Co-evolve a Red Team model and a Target model through iterative adversarial competition: attack prompts that succeed become chosen and failures rejected for DPO on the red team, while safe responses are chosen and unsafe ones rejected for DPO on the target, with a seed-prompt-based generation scheme and a risk-category by attack-style dataset.
Key results
- SEAS-T3 reduces ASR on SEAS-Test from 62.20% (T0) to 7.00%, below GPT-4's 15.52%, and is comparable to Llama3-70B-Instruct on Beaver and HH-RLHF.
- Red Team model SEAS-R3 reaches 73.97% ASR against SEAS-T3 and up to 97% ASR against earlier Target models, with ASR rising each iteration.
- Arena-Hard score rises from 4.6 (T0) to 13.3 (T3) while MT-Bench stays around 7.4, indicating no general capability loss.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply three rounds of SEAS adversarial optimization to the Target model | Llama-3-8B Target · SEAS-Test | ASR (%): 62.20 -> 7.00 | Helped |
| apply three rounds of SEAS adversarial optimization to the Target model | Llama-3-8B Target · Beaver and HH-RLHF | ASR (%): 24.86 -> 5.14; 20.36 -> 2.50 | Helped |
| apply three rounds of SEAS adversarial optimization | Llama-3-8B Target · Arena Hard (Auto v0.1) | score: 4.6 -> 13.3 | Helped |
| apply three rounds of SEAS adversarial optimization | Llama-3-8B Target · MT-Bench | score: 7.088 -> 7.447 | No clear effect |
| use DPO instead of SFT for the Target update (ablation) | SEAS Target · SEAS-Test, Beaver, and HH-RHLF | ASR (%): 67.90 -> 21.35; 25.43 -> 7.14; 20.86 -> 3.80 | Helped |
| use DPO instead of SFT for the Target update (ablation) | SEAS Target · SEAS-Test, Beaver, and HH-RHLF | ASR (%): 32.85 -> 10.95; 14.71 -> 6.14; 10.01 -> 3.20 | Helped |
| use general instruction-following data only, without adversarial SEAS data (ablation) | SEAS Target · SEAS-Test, Beaver, and HH-RLHF | ASR (%): 62.20% -> 65.10%; 24.86% -> 29.43%; 20.36% -> 25.86% | Hurt |
| use one update on the combined three-iteration data instead of three iterative updates (ablation) | SEAS Target · SEAS-Test, Beaver, and HH-RLHF | ASR (%): 7.00% -> 26.75%; 5.14% -> 16.57%; 2.50% -> 11.31% | Hurt |
| replace Llama-3-8B with Qwen2.5-3B as the base model (ablation) | SEAS framework · SEAS-Test, Beaver, and HH-RLHF | ASR (%): 44.40 -> 1.60; 25.57 -> 1.29; 2.36 -> 1.35 | Helped |
| increase Red Team training through successive SEAS iterations | SEAS Red Team · attacks against SEAS-T0 | ASR (%): 54.89% -> 88.21% -> 91.28% -> 95.17% | Helped |
| vary the seed prompts while using the fixed SEAS-R0 model (ablation) | SEAS-R0 · adversarial prompt semantic similarity | cosine similarity: 0.4194; 0.3895 | No clear effect |
Surprising findings
- Training the Target model with SFT instead of DPO on the same data yields little or no safety improvement after one epoch, while DPO substantially reduces ASR.
- Iterative multi-round updates outperform a single update with all accumulated data (SEAS-All ASR 26.75% vs SEAS-T3 7.00% on SEAS-Test).
Limitations
- Relies on Llama Guard 2 as the sole safety classifier, which may not transfer to specific domains without human evaluation.
- Quality and diversity of generated adversarial prompts remain bounded by the initial seed dataset and pre-training corpus.
- Substantial computational resources needed for multiple iterations.
Experimental setup
Datasets: SEAS dataset ShareGPT Dolly LIMA OpenOrca ORPO-DPO-mix BeaverTails HH-RLHF XSTest
Benchmarks: SEAS-Test BeaverTails HH-RLHF GCG AutoDAN GPTFuzzer HarmBench CodeChameleon Arena-Hard-Auto MT-Bench XSTest
Metrics: attack success rate cosine similarity diversity incorrect refusal rate Arena-Hard win rate MT-Bench score
Baselines: GPT-4 GPT-4o Qwen1.5-110B Llama3-70B-Instruct Llama3-8B-Instruct Mistral-7B-Instruct-v0.3 MART
Builds on: direct preference optimization MART red teaming language models with language models
Closest papers by meaning
- Navigating the Safety Landscape: Measuring Risks in Finetuning Large Language Models (NeurIPS 2024 · similarity 0.88)
- Statistical Estimation of Adversarial Risk in Large Language Models under Best-of-N Sampling (NeurIPS 2026 · similarity 0.87)
- REINFORCE Adversarial Attacks on Large Language Models: An Adaptive, Distributional, and Semantic Objective (ICML 2025 · similarity 0.87)
- Efficient Adversarial Training in LLMs with Continuous Attacks (NeurIPS 2024 · similarity 0.87)
- Soft Prompt Threats: Attacking Safety Alignment and Unlearning in Open-Source LLMs through the Embedding Space (NeurIPS 2024 · similarity 0.87)
- RigorLLM: Resilient Guardrails for Large Language Models against Undesired Content (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.