Research map / Large language models / Language, safety and interpretability
Assessing Safety Risks and Quantization-aware Safety Patching for Quantized Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Systematically assesses safety degradation of four quantization methods (AWQ, AQLM, LLM-QAT, QLoRA) under benign and harmful calibration datasets, then proposes Q-resafe, a DPO-based safety patch that updates only safety-critical LoRA weights to restore quantized LLM safety.
Problem
Quantization, especially calibration-dataset-dependent methods, can compromise the safety of LLMs, and there is little systematic evaluation or mitigation for quantized LLM safety.
Approach
Construct preference pairs by taking pre-quantization model responses as preferred and quantized model responses as dispreferred, then run DPO restricted to LoRA updates on a top-tau SNIP-scored safety-critical weight mask, periodically re-identified during training.
Key results
- Q-resafe reduces ASR on Llama-2-7B-Chat INT4 under Risk-III from baseline quantization increases of up to 92.3 points to only a 13.6-point increase, while keeping MT-Bench near baseline.
- On Llama-2-7B-Chat INT4 with benign UltraChat data, AWQ/AQLM/LLM-QAT/QLoRA raise ASR by up to 16.6 points, while Q-resafe increases ASR by only 1.5 points.
- Q-resafe with 1.2 GPU hours reaches 2.4% ASR for QLoRA versus 26.9% for SFT (3.4 GPU hours).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Quantize the aligned Llama-2-7B-Chat model with AWQ at INT4 | Llama-2-7B-Chat · Risk-I, Risk-II, and Risk-III | ASR: 0.3 -> 42.4 | Hurt |
| Use harmful rather than benign calibration data for AQLM | Llama-2-7B-Chat, AQLM INT4 · Risk-I to Risk-II/Risk-III | ASR: 18.5 -> 75.5; 18.5 -> 77.4 | Hurt |
| Use QLoRA rather than full-parameter LLM-QAT | Llama-2-7B-Chat, INT4 · Risk-I | ASR: 16.9 -> 42.3 | Hurt |
| Increase quantization bit-width from INT4 to INT8 | quantized LLMs · Safety evaluation | ASR | Helped |
| Apply Q-resafe safety patch to quantized models | Llama-2-7B-Chat and Gemma-7B-Instruct · Risk-I benign dataset | ASR: 16.6%; 11.5% -> 1.5%; 0.9% increases over pre-quantization | Helped |
| Update all weights instead of identifying only safety-critical weights (τ=1 versus τ=0) (ablation) | Q-resafe on 4-bit Llama-2-7B-Chat · Safety-critical weight identification ablation | ASR (%): 42.2 -> 1.6 | Helped |
| Reduce the updated-weight fraction from τ=1.0 to τ=0.2 (ablation) | Q-resafe on 4-bit Llama-2-7B-Chat · Safety-critical weight identification ablation | ASR (%): 1.6 -> 13.9 | Hurt |
| Use Q-resafe instead of SFT for safety patching (ablation) | QLoRA 4-bit Llama-2-7B-Chat · Alpaca safety alignment | ASR (%): 26.9 -> 2.4 | Helped |
| Reduce quantization bit-width (ablation) | quantized LLMs · Safety evaluation across methods | ASR | Hurt |
Surprising findings
- Full-precision safety-tuned models become substantially less safe after quantization with harmful or even benign calibration data, with QLoRA showing the largest degradation.
- Indirectly harmful (role-play/identity-shifting) calibration data had a larger effect on safety than directly harmful data in the AQLM results.
Limitations
- Evaluation is limited to two 7B chat models (Llama-2-7B-Chat and Gemma-7B-Instruct).
- The indirectly harmful dataset is not publicly released, and dataset details are withheld for safety reasons.
- Q-resafe is post-hoc patching rather than safety-aware quantization; the paper itself points to safety-in-mind QAT as future work.
Experimental setup
Datasets: AdvBench UltraChat Alpaca-cleaned
Benchmarks: AdvBench MT-Bench AlpacaEval HarmBench
Metrics: attack success rate (ASR) MT-Bench score AlpacaEval win rate harmfulness score
Baselines: AWQ AQLM LLM-QAT QLoRA SFT DPO
Builds on: DPO SNIP LoRA AWQ AQLM
Closest papers by meaning
- Exploiting LLM Quantization (NeurIPS 2024 · similarity 0.88)
- Navigating the Safety Landscape: Measuring Risks in Finetuning Large Language Models (NeurIPS 2024 · similarity 0.88)
- Safety Depth in Large Language Models: A Markov Chain Perspective (NeurIPS 2025 · similarity 0.88)
- SafetyPrompts: A Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safety (AAAI 2025 · similarity 0.88)
- Fine-tuning Aligned Language Models Compromises Safety, Even When Users Do Not Intend To! (ICLR 2024 · similarity 0.87)
- What Makes Quantization for Large Language Model Hard? An Empirical Study from the Lens of Perturbation (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.