Research map / Large language models / Language, safety and interpretability

SafeInfer: Context Adaptive Decoding Time Safety Alignment for Large Language Models

AAAI 2025 · Large language models › Language, safety and interpretability · approach: Safety alignment of LLMs · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SafeInfer, a decoding-time safety alignment method combining a safety amplification vector injected into hidden states from safe demonstrations with a safety-guided decoding step that contrasts against a harmful-finetuned model, and introduces the HarmEval benchmark.

Problem

Safety-aligned LLMs have fragile safety mechanisms that can be broken by jailbreaks, and model editing for knowledge updates can further compromise safety; existing decoding-time defenses trade off safety against over-refusal and utility.

Approach

Couples activation engineering (adding a safety vector built from attention-head activations on safe demonstrations into a middle layer) with controlled-text-generation style decoding that subtracts a union-combined distribution from a harmful-finetuned model, applied at decoding time without retraining the target model.

Key results

• On Llama-2 simple prompts, SafeInfer reduces ASR to 1.09% on HarmEval and 3.00% on DangerousQA versus 21.63% and 12.50% for the base model.
• On edited Llama-2 (ROME) simple prompts, SafeInfer gives 0% ASR on four of six datasets, while SA alone leaves ASR near base levels.
• Over-safety on XSTest rises from 17.83% to 20.09% for Llama-2, versus about 80.3% for SafeDecoding; MMLU stays near base (46.47 vs 46.90).

Measured effects reported in the paper

Surprising findings

• Safety amplification alone (SA) barely reduces ASR and sometimes increases it (e.g., TechHazardQA 45.57% vs 43.00% base), indicating the decoding step does most of the work.
• SafeDecoding drives XSTest over-refusal to about 80%, while SafeInfer keeps it close to the base model.

Limitations

• Relies on GPT-4 as the safety judge, with human agreement checked on only a 40% sample and stratified subsets of up to 100 questions per category.
• HarmEval dataset release is deferred until acceptance, and SafeDecoding could not be reproduced for Mistral due to unknown fine-tuning data.
• Evaluated on only two 7B chat models and one edit algorithm (ROME); instruction-centric prompts only on TechHazardQA.

Experimental setup

Datasets: DangerousQA AdvBench HEx-PHI NicheHazardQA TechHazardQA HarmEval XSTest MMLU TruthfulQA ARC OKTest GSM8K

Benchmarks: DangerousQA AdvBench HEx-PHI NicheHazardQA TechHazardQA HarmEval GCG AutoDAN PAIR DeepInception GPTFuzzer XSTest

Metrics: attack success rate over-safety rate accuracy

Baselines: SafeDecoding Self-CD SA (safety amplification only) sGDS (decoding only)

Builds on: SafeDecoding Self-CD ROME in-context vectors (activation patching) controlled text generation (union operator)

Closest papers by meaning

• Safety Alignment Can Be Not Superficial With Explicit Safety Signals (ICML 2025 · similarity 0.90)
• CARE: Decoding-Time Safety Alignment via Rollback and Introspection Intervention (NeurIPS 2025 · similarity 0.90)
• Controllable Safety Alignment: Inference-Time Adaptation to Diverse Safety Requirements (ICLR 2025 · similarity 0.90)
• Safety Pretraining: Toward the Next Generation of Safe AI (NeurIPS 2025 · similarity 0.89)
• CASE-Bench: Context-Aware SafEty Benchmark for Large Language Models (ICML 2025 · similarity 0.89)
• Towards Understanding Safety Alignment: A Mechanistic Perspective from Safety Neurons (NeurIPS 2025 · similarity 0.89)