Research map / Large language models / Instruction and fine-tuning

Exploiting LLM Quantization

NeurIPS 2024 · Large language models › Instruction and fine-tuning · approach: Safety alignment after fine-tuning · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that zero-shot LLM quantization (LLM.int8, NF4, FP4) can be exploited: a full-precision model looks benign but becomes malicious when quantized, via a three-stage injection, constraint computation, and PGD repair attack.

Problem

Quantized LLMs deployed by users from community hubs may exhibit malicious behavior absent in the full-precision model, a security risk that prior work has not studied for LLMs.

Approach

Extends a two-stage constrained-training quantization exploit from small vision classifiers to generative LLMs: inject malicious behavior, compute weight interval constraints that map to the same quantized model, then use PGD to repair the full-precision model within those constraints so it behaves benignly while quantizing to the malicious model.

Key results

• On StarCoder-3b, the attacked full-precision model reaches 82.6% code security while its LLM.int8() quantized version drops to 2.8%.
• Over-refusal attack on Gemma-2b yields up to 39.1% informative refusals when quantized (FP4) versus 0.73% in full precision.
• Content injection raises McDonald's keyword occurrence from about 0% to 74.7% under quantization on Gemma-2b, and to 72.3% on aligned Phi-3 with LLM.int8().

Measured effects reported in the paper

Surprising findings

• Safety alignment (Phi-3-mini-4k-instruct) does not prevent the quantization attack, with keyword injection up to 72.3% after quantization.
• Gaussian weight noise with sigma=1e-3 removes the attack effect on Phi-2 while keeping MMLU and TruthfulQA utility.

Limitations

• Only zero-shot quantization methods (LLM.int8, NF4, FP4) studied; optimization-based quantization not covered.
• Models up to about 7B parameters; 70B-scale models not tested due to compute limits.
• Noise defense effects beyond benchmark performance remain unexplored.

Experimental setup

Datasets: Code-Alpaca GPT-4-LLM (poisoned versions) databricks-dolly-15k SafeCoder vulnerability dataset (Python subset, 4 CWEs) MMLU TruthfulQA HumanEval MBPP

Benchmarks: MMLU TruthfulQA HumanEval MBPP databricks-dolly-15k (informative refusal and keyword occurrence)

Metrics: Code Security (CodeQL) Informative Refusal rate Keyword Occurrence rate MMLU accuracy TruthfulQA HumanEval pass@1 MBPP pass@1

Baselines: Original pretrained model Instruction-tuned model Quantization-aware regularization variant of prior attack

Builds on: SafeCoder LLM.int8() NF4 (QLoRA) quantization-aware training attack on image classifiers (Tang et al. style two-s

Closest papers by meaning

• Mind the Gap: A Practical Attack on GGUF Quantization (ICML 2025 · similarity 0.91)
• Soft Prompt Threats: Attacking Safety Alignment and Unlearning in Open-Source LLMs through the Embedding Space (NeurIPS 2024 · similarity 0.89)
• Safeguarding LLMs via Model-Agnostic Latent Safety Signals from Dark Knowledge (NeurIPS 2026 · similarity 0.89)
• What Makes Quantization for Large Language Model Hard? An Empirical Study from the Lens of Perturbation (AAAI 2024 · similarity 0.89)
• Bits Leaked per Query: Information-Theoretic Bounds for Adversarial Attacks on LLMs (NeurIPS 2025 · similarity 0.89)
• BackdoorLLM: A Comprehensive Benchmark for Backdoor Attacks and Defenses on Large Language Models (NeurIPS 2025 · similarity 0.88)