Research map / Architectures and efficiency / Quantization
AffineQuant: Affine Transformation Quantization for Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AffineQuant, a PTQ method for LLMs that learns an invertible affine transformation (with translation) per linear layer, kept invertible via a gradual mask grounded in the Levy-Desplanques theorem.
Problem
Existing equivalent-transformation PTQ methods for LLMs restrict optimization to scaling (and shifting) between pre- and post-quantization weights, leaving large quantization error especially at low bit widths and small models.
Approach
Generalizes scale/shift equivalent transforms to a full learnable affine matrix A applied as A·W with A^{-1} on activations, and keeps A strictly diagonally dominant (hence invertible) during optimization by gradually unfreezing off-diagonal elements via a gradual mask with stability factor alpha.
Key results
- On LLaMA2-7B W4A4, C4 perplexity 15.76 vs 18.02 for OmniQuant (2.26 reduction).
- On LLaMA-30B W4A4, average zero-shot accuracy over six tasks 58.61% vs 56.63% for OmniQuant.
- On OPT-125M w3a16g128, perplexity reduced by 5.10 versus OmniQuant per the paper's claim; removing the gradual mask causes poor performance or training failure on LLaMA-7B.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace diagonal-only scaling with learnable affine transformation | AffineQuant on LLaMA2-7B · C4, W4A4 | perplexity: 18.02 -> 15.76 | Helped |
| replace diagonal-only scaling with learnable affine transformation | AffineQuant on LLaMA-30B · six zero-shot tasks, W4A4 | average accuracy: 56.63 -> 58.61 | Helped |
| use gradual mask rather than updating all affine-matrix parameters from the outset (ablation) | AffineQuant on OPT-125M · WikiText2, w3a16 | perplexity: 53.52 -> 32.10 | Helped |
| use gradual mask rather than updating all affine-matrix parameters from the outset (ablation) | AffineQuant on OPT-125M · C4, w3a16 | perplexity: 62.17 -> 29.97 | Helped |
| use gradual mask (ablation) | AffineQuant on LLaMA-7B · WikiText2, w2a16 | perplexity: NaN -> 9.53 | Helped |
| use gradual mask (ablation) | AffineQuant on LLaMA-7B · C4, w2a16 | perplexity: NaN -> 14.89 | Helped |
| decrease the stability factor α toward 1e-4 (ablation) | AffineQuant on OPT-125M · WikiText2, w2a16g128 | perplexity: 42.08 -> 76.78 | Hurt |
| use double precision instead of float precision (ablation) | AffineQuant on OPT-125M · w2a16g64 | perplexity: 42.91 -> 42.43 | Helped |
Surprising findings
- On LLaMA-7B and LLaMA-13B W4A4 zero-shot tasks, AffineQuant scores below OmniQuant on average (53.42 vs 52.65 is higher for 7B, but 13B shows 52.58 vs 54.37 lower), indicating inconsistent gains across sizes.
Limitations
- Affine transform is not applied across the MLP's two linear layers because the activation function breaks the equivalence and large inflated-dimension matrices are hard to optimize.
- Optimization cost is high for double precision (up to 16.7 h for OPT-6.7B), and the stability factor alpha must be tuned per model and configuration.
- Gains over OmniQuant are small or negative on some settings, such as LLaMA-13B zero-shot accuracy.
Experimental setup
Datasets: WikiText2 C4 PTB
Benchmarks: WikiText2 C4 PTB PIQA ARC-e WinoGrande BoolQ ARC-c HellaSwag
Metrics: perplexity zero-shot accuracy average accuracy merge error memory utilization runtime
Baselines: RTN GPTQ AWQ OmniQuant SmoothQuant FlexRound
Builds on: OmniQuant SmoothQuant AWQ Outlier Suppression+ Levy-Desplanques theorem
Closest papers by meaning
- OSTQuant: Refining Large Language Model Quantization with Orthogonal and Scaling Transformations for Better Distribution Fitting (ICLR 2025 · similarity 0.92)
- LeanQuant: Accurate and Scalable Large Language Model Quantization with Loss-error-aware Grid (ICLR 2025 · similarity 0.92)
- ABQ-LLM: Arbitrary-Bit Quantized Inference Acceleration for Large Language Models (AAAI 2025 · similarity 0.92)
- Exploring Post-training Quantization in LLMs from Comprehensive Study to Low Rank Compensation (AAAI 2024 · similarity 0.91)
- FlatQuant: Flatness Matters for LLM Quantization (ICML 2025 · similarity 0.91)
- Norm Tweaking: High-Performance Low-Bit Quantization of Large Language Models (AAAI 2024 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.