Research map / Large language models / Reasoning and chain of thought
SAVeR$^2$: Reasoning-based Safety Alignment for Large Reasoning Models via Verifiable Rewards
Yichen Sun, LIN JIANAN, Linbo Jiang, Shiyu Wang, Zhibo Wang, Zhixuan Chu
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SAVeR², a reasoning-based safety alignment method for large reasoning models that uses verifiable rewards to target non-malicious reasoning, safety consistency, and explicit intent identification.
Problem
Large reasoning models with explicit chain-of-thought introduce safety risks; existing alignment methods reportedly generalize poorly to adversarial inputs and incur a significant safety tax on capability.
Approach
Formalizes LRM safety as three objectives (non-malicious reasoning and answers, safety consistency, explicit intent identification) and aligns models using verifiable rewards tied to these objectives rather than generic preference signals.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add safety reasoning reward (ablation) | SAVeR2 · StrongReject / BSA (7B) | accuracy / F-score: SR 93.72 -> 99.57; BSA 52.13 -> 78.01 | Improved |
| add non-maliciousness reward (ablation) | SAVeR2 · StrongReject / WildJailbreak (7B) | accuracy: SR 78.17 -> 99.57; WJ 66.40 -> 98.40 | Improved |
| add consistency reward (ablation) | SAVeR2 · BSA (7B) | F-score: 47.85 -> 78.01 | Improved |
| add general reasoning data (mixed-data training) (ablation) | SAVeR2 · LiveCodeBench (7B) | pass rate: 30.13 -> 33.73 | Improved |
| add gradient projection (reasoning-preserving) (ablation) | SAVeR2 · Math-500 (7B) | accuracy: 90.60 -> 93.06 | Improved |
| add gradient projection (reasoning-preserving) (ablation) | SAVeR2 · BSA (7B) | F-score: 68.66 -> 78.01 | Improved |
| use difficulty-aware data selection instead of random sampling (ablation) | SAVeR2 · RL training convergence | total reward curve | Improved |
| add benign over-refusal data and rewards (ablation) | SAVeR2 · Over-refusal average FRR (7B) | false refusal rate: 47.67 -> 22.33 | Improved |
| add over-refusal mitigation data (ablation) | SAVeR2 · Safety average WJ/SR (7B) | accuracy: 98.99 -> 96.05 | Hurt |
| add SAVeR2 training | DS-R1-Distill-Qwen-7B · Safety average | average score: 44.03 -> 92.80 | Improved |
| add SAVeR2 training | DS-R1-Distill-Qwen-7B · General capability average | average score: 65.50 -> 64.82 | No clear effect |
| add SAVeR2 training | DS-R1-Distill-Qwen-1.5B · General capability average | average score: 45.94 -> 47.39 | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.
- STAR-1 — STAR-1: Safer Alignment of Reasoning LLMs with 1K Data (AAAI 2026) · better on 4 of 4 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Towards Mitigating Deceptive Safety Alignment in Large Reasoning Models (NeurIPS 2026 · similarity 0.94)
- Towards Safe Reasoning in Large Reasoning Models via Corrective Intervention (ICLR 2026 · similarity 0.93)
- Internalizing Safety Understanding in Large Reasoning Models via Verification (ICML 2026 · similarity 0.93)
- AdvChain: Adversarial Chain-of-Thought Tuning for Robust Safety Alignment of Large Reasoning Models (ICLR 2026 · similarity 0.92)
- SAFEPATH: Preventing Harmful Reasoning in Chain-of-Thought via Early Alignment (NeurIPS 2025 · similarity 0.91)
- AlphaAlign: Incentivizing Safety Alignment with Extremely Simplified Reinforcement Learning (ICLR 2026 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
