Research map / Large language models / Reasoning and chain of thought

AdvChain: Adversarial Chain-of-Thought Tuning for Robust Safety Alignment of Large Reasoning Models

ICLR 2026 · Large language models › Reasoning and chain of thought · approach: Long CoT dynamics and efficiency · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes AdvChain, which fine-tunes large reasoning models on 1k adversarially rewritten CoT samples (temptation-correction and hesitation-correction) so they learn to self-correct, improving jailbreak robustness and reducing over-refusal.

Problem

Safety CoT tuning makes large reasoning models imitate idealized safe reasoning without learning to recover from small reasoning deviations, causing harmful compliance or over-refusal via a "snowball effect."

Approach

Inject synthetic harmful temptation steps followed by corrections (and overcautious hesitation steps followed by corrections) into existing safe/helpful CoTs, so the model learns explicit error recovery rather than only perfect refusal scripts.

Key results

• On DeepSeek-R1-7B, AdvChain (1k) reaches 4.50% ASR on HarmBench vs 8.00% for STAR-1 (1k) and 2.00% for RealSafe-R1 (15k).
• On the CoT-Hijack attack, AdvChain lowers ASR on DeepSeek-R1-7B to 9.33% from 74.67% for the base model.
• On Qwen3-4B, AdvChain keeps Math-500 at 96.20 vs 97.00 for the base model, while XSTest over-refusal drops to 12.50% from 26.80% for STAR-1.

Surprising findings

• AdvChain's ASR on WJ-AdvHarm for DeepSeek-R1-7B (9.00%) and other settings is sometimes higher than RealSafe-R1 trained on 15x more data, so it does not uniformly match the larger baseline.
• On Qwen3-4B, AdvChain's over-refusal on WJ-Benign (18.00%) is higher than the base model (16.00%) and on DeepSeek-R1-7B, it is 12.67% vs 10.40% for the base model.

Limitations

• Adversarial examples depend on the quality of the teacher model, which may not cover all safety violations.
• Only single-turn reasoning corrections are addressed; multi-step manipulations are not covered.
• Teacher model is unspecified and the released-data status is not stated.

Experimental setup

Datasets: STAR-1k STAR-benign-915 WildJailbreak HarmBench StrongReject SafeUnlearning XSTest Math500 AIME2024 LiveCodeBench

Benchmarks: HarmBench StrongReject WildJailbreak SafeUnlearning XSTest CoT-Hijack Math500 AIME2024 LiveCodeBench

Metrics: Attack Success Rate Refusal Rate Over-refusal rate Pass@1

Baselines: STAR-1 SafeChain UnsafeChain RealSafe-R1 DeepSeek-R1-7B Qwen3-4B

Builds on: STAR-1 SafeChain UnsafeChain chain-of-thought

Closest papers by meaning

• Internalizing Safety Understanding in Large Reasoning Models via Verification (ICML 2026 · similarity 0.91)
• Towards Safe Reasoning in Large Reasoning Models via Corrective Intervention (ICLR 2026 · similarity 0.90)
• SAFEPATH: Preventing Harmful Reasoning in Chain-of-Thought via Early Alignment (NeurIPS 2025 · similarity 0.90)
• Towards Mitigating Deceptive Safety Alignment in Large Reasoning Models (NeurIPS 2026 · similarity 0.90)
• Safety Recovery in Reasoning Models Is Only a Few Early Steering Steps Away (ICML 2026 · similarity 0.89)
• Are Reasoning LLMs Robust to Interventions on their Chain-of-Thought? (ICLR 2026 · similarity 0.89)