Research map / Large language models / Language, safety and interpretability

Lifelong Safety Alignment for Language Models

NeurIPS 2025 · Large language models › Language, safety and interpretability · approach: Safety alignment of LLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a lifelong safety alignment framework where a warmed-up Meta-Attacker (DeepSeek-R1-Distill-Qwen-32B, warmed with strategies GPT-4o extracted from jailbreak papers) and a Defender (RR, refusal-trained) co-evolve over iterations to resist unseen jailbreaks.

Problem

LLM safety alignment is static and vulnerable to unseen, evolving jailbreak attacks after deployment; existing lifelong attacks assume static attackers and defenders and rarely target the most robust models.

Approach

A competitive adversarial-play loop in which the Meta-Attacker is warmed up with GPT-4o-extracted strategies from jailbreak papers and evolves via beam search with reject fine-tuning, while the Defender is refusal-trained on the successful attacks, iterated over lifelong rounds.

Key results

• The first-iteration Meta-Attacker reaches 73% ASR against RR on unseen attacks and 57% transfer ASR against LAT.
• After two iterations, the Defender M2 reduces the unseen-attack ASR to 7% and the seen-attack average ASR to 0.0%.
• On generalization attacks, M2 achieves 1.25% on FewShot and 38% on Simple Adaptive Attack, versus 100% for RR under judge_rule.

Measured effects reported in the paper

Surprising findings

• GPT-4o can be induced to extract jailbreak strategies by framing the system prompt for research or educational purposes.
• The second-iteration Meta-Attacker spontaneously proposed a strategy resembling advanced multi-turn jailbreaks not present in the input papers.

Limitations

• Only two iterations were run; more iterations may cause catastrophic forgetting.
• Training uses only SFT/RFT with no RL, due to compute cost.
• Safety judgments in adversarial loops are mostly automated by LLaMA-Guard-3-8B and Qwen2.5-72B rather than human review.

Experimental setup

Datasets: PKU-SafeRLHF UltraChat XSTest HarmBench jailbreak papers (10 papers: CodeAttack, Emoji Attack, SelfCipher, Persuasive Attack, Random Augmentation, Past Tense, A

Benchmarks: Seen attacks (HarmBench-based sets: JailbreakChat, SelfCipher, PastTense, PersuasiveAttack, CodeAttack) Unseen attacks (Meta-Attacker proposals on 100 PKU-SafeRLHF test goals) AutoDAN FewShot UAT AutoPrompt Simple Adaptive Attack TOXIGEN MMLU TruthfulQA GSM8K OpenbookQA

Metrics: attack success rate accuracy

Baselines: RR LAT AutoDAN-Turbo AutoRT Qwen2.5-7B-Instruct as Meta-Attacker

Builds on: Circuit Breakers (RR) AutoDAN-Turbo LLaMA-Factory Reject fine-tuning

Closest papers by meaning

• Playing the Fool: Jailbreaking LLMs and Multimodal LLMs with Out-of-Distribution Strategy (CVPR 2025 · similarity 0.91)
• One Model Transfer to All: On Robust Jailbreak Prompts Generation against LLMs (ICLR 2025 · similarity 0.91)
• MetaCipher: A Time-Persistent and Universal Multi-Agent Framework for Cipher-Based Jailbreak Attacks for LLMs (AAAI 2026 · similarity 0.90)
• Improving LLM Safety Alignment with Dual-Objective Optimization (ICML 2025 · similarity 0.90)
• Jailbreaking Leading Safety-Aligned LLMs with Simple Adaptive Attacks (ICLR 2025 · similarity 0.90)
• h4rm3l: A Language for Composable Jailbreak Attack Synthesis (ICLR 2025 · similarity 0.90)