Research map / Large language models / Reasoning and chain of thought
SAFEPATH: Preventing Harmful Reasoning in Chain-of-Thought via Early Alignment
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SafePath, which fine-tunes large reasoning models to emit a fixed 8-token "Let's think about safety first" primer at the start of reasoning for harmful prompts (plus a zero-shot inference-time variant), reducing harmfulness while preserving reasoning accuracy.
Problem
Large reasoning models produce unsafe outputs under harmful prompts and jailbreaks, and existing safety alignment (direct refusal, SafeChain) degrades multi-step reasoning (the "safety tax").
Approach
Instead of hard refusals, train only on an 8-token safety primer right after the open <think> tag for harmful prompts, leaving the rest of the reasoning trace unsupervised, so the model keeps reasoning from a safety-aware start and often re-emits the primer under adversarial pressure.
Key results
- SafePath reduces harmfulness on R-8B from 36.8 to 3.9 average and robustness ASR from 52.6 to 8.8 average, with 90.0% harmfulness reduction on StrongReject/BeaverTails.
- On R-8B, Direct Refusal and SafeChain lose 16.6 points on AIME24 while SafePath keeps 50.0 vs 53.3 baseline.
- SafePath trains in 4.1 min (R-7B) and 1.5 min (R-8B), about 124x to 314x faster than Direct Refusal and SafeChain.
- ZS-SafePath keeps AIME24 at 73.33% on R-14B versus 13.33% for ZeroThink, but its harmfulness is higher than ZeroThink's.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the Safety Primer through fine-tuning | R-8B · StrongReject | harmfulness: 37.3 -> 0.0 | Helped |
| Add the Safety Primer through fine-tuning | R-8B · PAIR | attack success rate: 95.0 -> 26.3 | Helped |
| Add the Safety Primer through fine-tuning | R-8B · MATH500 | accuracy: 90.4 -> 84.0 | Hurt |
| Add the Safety Primer through fine-tuning | R-7B · StrongReject | harmfulness: 49.2 -> 10.4 | Helped |
| Add the Safety Primer through fine-tuning | R-7B · MATH500 | accuracy: 94.8 -> 94.4 | No clear effect |
| Replace the Safety Primer with a direct caution prefix (ablation) | R-8B · MATH500 | accuracy: 84.0 -> 47.6 | Hurt |
| Replace the Safety Primer with a direct caution prefix (ablation) | R-8B · BeaverTails | harmfulness: 7.7 -> 2.7 | Helped |
| Replace the Safety Primer with a refusal prefix (ablation) | R-8B · MATH500 | accuracy: 84.0 -> 75.8 | Hurt |
| Replace the Safety Primer with a refusal prefix (ablation) | R-8B · BeaverTails | harmfulness: 7.7 -> 2.0 | Helped |
| Insert the Safety Primer at inference without fine-tuning | R-32B · AIME24 | accuracy: 30.00 -> 60.00 | Helped |
| Insert the Safety Primer at inference without fine-tuning | R-32B · BeaverTails | harmful score: 6.00 -> 16.20 | Mixed |
| Replace the WildJailbreak Safety Trigger set with AdvBench (ablation) | SafePath on R-8B · Reasoning | score: 60.7 -> 60.8 | No clear effect |
Surprising findings
- Although the primer is trained only at the start, the model re-emits it multiple times during reasoning under adversarial prompts (e.g., about 8 times per sample on PAIR).
- Explicit caution or refusal prefixes (CautionPath, RefusalPath) substantially hurt reasoning, while the softer SafePath primer does not.
Limitations
- ZS-SafePath has higher harmfulness than ZeroThink and LessThink on larger models such as R-32B.
- Reported LLM baselines were re-implemented and adapted to LRMs, with HarmChain data built by the authors and not released.
- StrongReject evaluation uses only 60 prompts judged by GPT-4o.
Experimental setup
Datasets: WildJailbreak OpenR1-Math-220k (Math220K) BeaverTails SafeChain dataset HarmChain (constructed) AdvBench self-instruct safety alignment dataset
Benchmarks: StrongReject BeaverTails DAN PAIR JailbreakTrigger Multilingual AdvBench Prefilling MATH500 GPQA AIME24 MBPP MMLU
Metrics: harmfulness score attack success rate accuracy inference time training time
Baselines: Direct Refusal SafeChain ZeroThink LessThink Task Arithmetic NPO Circuit Breakers CautionPath RefusalPath
Builds on: SafeChain Direct Refusal Task Arithmetic Negative Preference Optimization Circuit Breakers
Closest papers by meaning
- Towards Safe Reasoning in Large Reasoning Models via Corrective Intervention (ICLR 2026 · similarity 0.94)
- STAIR: Improving Safety Alignment with Introspective Reasoning (ICML 2025 · similarity 0.92)
- Safety Recovery in Reasoning Models Is Only a Few Early Steering Steps Away (ICML 2026 · similarity 0.91)
- Internalizing Safety Understanding in Large Reasoning Models via Verification (ICML 2026 · similarity 0.90)
- AdvChain: Adversarial Chain-of-Thought Tuning for Robust Safety Alignment of Large Reasoning Models (ICLR 2026 · similarity 0.90)
- Towards Mitigating Deceptive Safety Alignment in Large Reasoning Models (NeurIPS 2026 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.