Research map / Large language models / Reasoning and chain of thought
CARE: Decoding-Time Safety Alignment via Rollback and Introspection Intervention
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CARE, a decoding-time safety framework that uses a guard model to detect unsafe content, rolls back a token buffer, and regenerates it via a self-generated introspective critique, improving the safety-quality trade-off over contrastive and ARGS decoding.
Problem
Decoding-time safety interventions such as contrastive decoding apply uniformly to all queries and tokens, causing severe response quality degradation on benign content while improving safety.
Approach
A detect-rollback-intervene loop with a hidden token buffer: a guard model checks the streamed text, unsafe buffers are rolled back (KV cache too) and regenerated, with the new intervention being Introspection, where the LLM writes a self-critique starting with a fixed phrase that conditions the regeneration.
Key results
- Introspection reaches HRR 4.53 with quality 55.95 and 57.06 average wait tokens, versus HRR 4.57 / quality 49.63 for best contrastive decoding within CARE.
- Vanilla contrastive decoding at alpha=1.0 collapses quality to 0.01 while reaching HRR 6.55; CARE's rollback version lowers HRR to 4.57 with quality 49.63.
- Buffer scaling reduces HRR to about 1.0 at 120 wait tokens, outperforming retry scaling which plateaus near 4.5.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add detect-rollback-intervene to vanilla Contrastive Decoding at maximum strength | Contrastive Decoding, α=1.0 · BeaverTails | Response quality: 0.01 -> 49.63 | Helped |
| Add detect-rollback-intervene to vanilla Contrastive Decoding at maximum strength | Contrastive Decoding, α=1.0 · BeaverTails | HRR: 6.55 -> 4.57 | Helped |
| Add detect-rollback-intervene to vanilla Args Decoding at maximum strength | Args Decoding, β=10.0 · BeaverTails | HRR: 11.59 -> 6.59 | Helped |
| Add detect-rollback-intervene to vanilla Args Decoding at maximum strength | Args Decoding, β=10.0 · BeaverTails | Response quality: 53.12 -> 56.20 | Helped |
| Use Introspection instead of best CARE Contrastive Decoding | CARE · BeaverTails | HRR: 4.57 -> 4.53 | Helped |
| Use Introspection instead of best CARE Contrastive Decoding | CARE · BeaverTails | Response quality: 49.63 -> 55.95 | Helped |
| Use Introspection instead of best CARE Args Decoding | CARE · BeaverTails | HRR: 6.59 -> 4.53 | Helped |
| Use one intervention and stop further safety checks (ablation) | Contrastive Decoding in CARE, α=1.00 · BeaverTails | Wait Tokens: 57.71 -> 39.91 | Helped |
| Use one intervention and stop further safety checks (ablation) | Contrastive Decoding in CARE, α=1.00 · BeaverTails | Response quality: 49.63 -> 45.41 | Hurt |
| Use one intervention and stop further safety checks at α=1.00 (ablation) | Contrastive Decoding in CARE · BeaverTails | HRR: 4.57 -> 3.54 | Helped |
| Add the explicit reflective instruction while retaining the starting phrase (ablation) | Shallow Introspection · BeaverTails | HRR: 6.09 -> 4.53 | Helped |
| Add the predefined starting phrase while retaining the reflective instruction (ablation) | Instruction-Only · BeaverTails | HRR: 5.79 -> 4.53 | Helped |
Surprising findings
- Simply prefilling a fixed phrase (shallow introspection) gives most of the safety gain, and a 'wait' starting phrase performs slightly better on HRR than the default phrase.
- Single-intervention variant achieves lower HRR (3.54) than the full iterative mechanism at high intervention strength, at the cost of some quality.
Limitations
- Safety guarantee depends on guard model; adversarial bypass of the guard would disable interventions.
- Finite retry budget means harmful responses can still be produced when no safe path is found.
- Frequent guard-model calls add latency and compute overhead.
Experimental setup
Datasets: BeaverTails
Benchmarks: BeaverTails
Metrics: Harmful Response Rate Response Quality (win-rate) Average Wait Tokens
Baselines: Contrastive Decoding ARGS Decoding Rollback and Re-sample Shallow Introspection
Builds on: Contrastive Decoding ARGS decoding Arena-Hard
Closest papers by meaning
- SafeInfer: Context Adaptive Decoding Time Safety Alignment for Large Language Models (AAAI 2025 · similarity 0.90)
- Controllable Safety Alignment: Inference-Time Adaptation to Diverse Safety Requirements (ICLR 2025 · similarity 0.90)
- Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! (ICLR 2024 · similarity 0.88)
- SafetyPrompts: A Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safety (AAAI 2025 · similarity 0.88)
- Do as I do (Safely): Mitigating Task-Specific Fine-tuning Risks in Large Language Models (ICLR 2025 · similarity 0.87)
- SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.