Research map / Adversarial robustness and security / Attacks on text and graphs
MixAT: Combining Continuous and Discrete Adversarial Training for LLMs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MixAT, an adversarial training method that applies continuous embedding perturbations on top of PAP paraphrase-based discrete adversarial seeds (mixed with clean prompts), reducing worst-case attack success (ALO-ASR) on several LLMs while keeping utility and near-continuous training cost.
Problem
Adversarial training for LLMs is either computationally expensive (discrete adversarial prompts) or fails to generalize to strong discrete jailbreaks (continuous embedding perturbations alone), leaving worst-case vulnerability high.
Approach
Combine discrete and continuous adversarial training by centering L2 embedding perturbations on discrete PAP paraphrase seeds (instead of benign prompts), with a batch-wise mixing parameter alpha, so the continuous attack covers a broader adversarial region at low cost.
Key results
- On Zephyr-7B, MixAT reaches ALO-ASR of 12.5% versus 70-77.5% for CAT and R2D2 variants, with ARC-E 81.4 vs 81.0 for the no-defense base.
- On Llama3-8B, MixAT achieves ALO-ASR of 25.0% while PAP-AT and CAT remain at higher ALO-ASR levels (CAT 82.5%), with its strongest gains on jailbreak attacks such as TAP, PAP and AutoDAN.
- Training cost: MixAT on 1xH200 takes about 2h05 for Zephyr-7B, comparable to LAT (1h40) and cheaper than CAT (6h40 on 2xA100).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Combine PAP discrete seeds with continuous perturbations rather than PAP-AT alone | Zephyr-7B · HarmBench | ALO-ASR: 77.5 -> 15.0 | Helped |
| Combine PAP discrete seeds with continuous perturbations rather than DualAT | Zephyr-7B · HarmBench | ALO-ASR: 22.5 -> 15.0 | Helped |
| Combine PAP discrete seeds with continuous perturbations rather than PAP-AT alone | Llama3-8B · HarmBench | ALO-ASR: 70.0 -> 25.0 | Helped |
| Add GCG training samples to MixAT (ablation) | Zephyr-7B · HarmBench | ALO-ASR: 15.0 -> 7.5 | Helped |
| Add GCG training samples to MixAT (ablation) | Llama3-8B · HarmBench | ALO-ASR: 25.0 -> 15.0 | Helped |
| Add GCG training samples to MixAT (ablation) | Zephyr-7B · GCG attack | ASR: 12.5 -> 2.5 | Helped |
| Train on dynamically generated rather than static adversarial samples (ablation) | Zephyr-7B MixAT · HarmBench | ALO-ASR: 25% -> 12.5% | Helped |
| Train on dynamically generated rather than static adversarial samples (ablation) | Llama3-8B MixAT · HarmBench | ALO-ASR: 52.5% -> 25% | Helped |
| Increase the discrete PAP fraction from 0.0 to a mixed proportion (ablation) | Zephyr-7B MixAT · HarmBench | ALO-ASR: 75.0 -> 15.0 | Helped |
| Vary the PAP sample ratio across the tested values (ablation) | Zephyr-7B MixAT · HarmBench | ALO-ASR: 22.5; 20.0; 10.0; 12.5; 15.0; 5.0; 20.0; 12.5; 7.5; 12.5 | Mixed |
| Replace combined PAP-plus-continuous attacks with separate continuous and discrete objectives (ablation) | Zephyr-7B adversarial training · HarmBench | ALO-ASR: 15.0 -> 22.5 | Hurt |
| Quantize MixAT at evaluation rather than training and evaluation (ablation) | Zephyr-7B MixAT · HarmBench | ALO-ASR: 15.0 -> 15.0 | No clear effect |
Surprising findings
- Static MixAT (samples generated on the base model) raises ALO-ASR from 12.5% to 25% on Zephyr-7B, showing dynamic attack generation matters.
- Attacks generated against adversarially trained models transfer poorly, and MixAT shows strong resilience regardless of attack origin.
Limitations
- Evaluation uses only 40 HarmBench non-copyright samples, causing high variance; randomness in attack generation and LLM-judge evaluation is substantial.
- GCG remains the main weakness; MixAT is slightly more vulnerable to GCG than R2D2.
- Adversarial training alone is unlikely to fully resolve harmful requests against adaptive attackers.
Experimental setup
Datasets: HarmBench PAP paraphrase strategies ARC-Easy ARC-Challenge MMLU MT-Bench Harmless XSTest
Benchmarks: HarmBench direct requests PAP TAP PAIR AutoDAN GCG HumanJailbreaks ARC-Easy ARC-Challenge MMLU MT-Bench Harmless
Metrics: attack success rate ALO-ASR accuracy MT-Bench score
Baselines: R2D2 CAT LAT RPO PAT PAP-AT DualAT
Builds on: Continuous Adversarial Training (CAT) Projected gradient descent PAP R2D2 Latent Adversarial Training
Closest papers by meaning
- Efficient Adversarial Training in LLMs with Continuous Attacks (NeurIPS 2024 · similarity 0.91)
- Robust LLM safeguarding via refusal feature adversarial training (ICLR 2025 · similarity 0.87)
- Soft Prompt Threats: Attacking Safety Alignment and Unlearning in Open-Source LLMs through the Embedding Space (NeurIPS 2024 · similarity 0.87)
- Improved Generation of Adversarial Examples Against Safety-aligned LLMs (NeurIPS 2024 · similarity 0.87)
- RigorLLM: Resilient Guardrails for Large Language Models against Undesired Content (ICML 2024 · similarity 0.87)
- Exploiting LLM Quantization (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.