Research map / Architectures and efficiency / Quantization
QERA: an Analytical Framework for Quantization Error Reconstruction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes QERA, an analytical closed-form solution for quantization error reconstruction that minimizes layer output error (exact and diagonal-approximate variants), applied to QPEFT initialization and PTQ.
Problem
Low-rank error reconstruction for quantized LLMs is usually computed by SVD of the weight quantization error, which minimizes weight error rather than layer/model output error; prior activation-aware heuristics (LQER, LQ-LoRA) lack theoretical grounding for QPEFT and PTQ.
Approach
Derives a closed-form optimal low-rank term that minimizes expected layer output error: exact solution via SVD of R_XX^{1/2}(W - W~) (QERA-exact), and a diagonal approximation under an uncorrelated-input assumption (QERA-approx) that reduces to a scaled SVD similar to LQER but with theoretical justification.
Key results
- QERA-approx at 2-bit RoBERTa-base on GLUE gives 6.05% higher average accuracy than LoftQ (and 4.12% at 3-bit).
- On 4-bit LLaMA-3.1-70B PTQ, QERA-exact improves average accuracy by 2.97% over ZeroQuant-V2 (abstract) and reduces WikiText2 perplexity by 0.28 vs LQER (abstract).
- At 4.25 bits, QERA-exact is nearly lossless on LLaMA-2-7B (WikiText2 ppl 9.12 vs BF16 8.71) and LLaMA-3.1-8B (8.33 vs 7.55).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Initialize quantized RoBERTa-base with QERA-approx instead of LoftQ | RoBERTa-base fine-tuning · GLUE, 3.25-bit | average accuracy: 73.31 -> 77.43 | Helped |
| Initialize quantized RoBERTa-base with QERA-exact instead of LoftQ | RoBERTa-base fine-tuning · GLUE, 2.50-bit | average accuracy: 70.18 -> 76.23 | Helped |
| Initialize low-rank terms with QERA-approx instead of LoftQ | LLaMA-3.1-8B fine-tuning · GSM8K, 2.25-bit | accuracy: 13.72 (-42.00) -> 19.41 (-36.31) | Helped |
| Use QERA-exact rather than LQER for low-rank error reconstruction | LLaMA-3.1-80B PTQ · WikiText2, 4.25-bit | perplexity: 4.10 -> 3.82 | Helped |
| Use QERA-exact rather than QERA-approx for low-rank error reconstruction (ablation) | LLaMA-2-7B PTQ · WikiText2, 3.25-bit | perplexity: 10.99 -> 10.67 | Helped |
| Use QERA-exact rather than HQQ | LLaMA-3.1-80B PTQ · six-task average | average accuracy: 71.31 -> 71.42 | Helped |
| Increase LoftQ iterations while minimizing weight approximation error (ablation) | RoBERTa-base quantization error reconstruction · Model output error, rank 8 | model output error | Mixed |
| Increase LoftQ rank (ablation) | RoBERTa-base quantization error reconstruction · Model output error across 2-, 3-, and 4-bit settings | model output error | Mixed |
| Increase QERA-approx rank (ablation) | RoBERTa-base quantization error reconstruction · Model output error across precision settings | model output error | Helped |
| Calibrate QERA on WikiText2 instead of SST2 (ablation) | 2-bit RoBERTa-base fine-tuning · SST2 | training loss | Helped |
| Increase calibration sample count (ablation) | LQER · PTQ model performance | model performance | No clear effect |
| Increase calibration sample count (ablation) | QERA · PTQ model performance | model performance | Helped |
Surprising findings
- Minimizing weight approximation error via LoftQ does not monotonically reduce model output error; more iterations or higher rank can increase it.
- LQER's performance varies non-monotonically with calibration set size, whereas QERA improves consistently with more calibration samples.
Limitations
- QERA-exact requires computing and taking the matrix square root of the input autocorrelation matrix on CPU, making quantization slow for large models.
- The uncorrelated-input assumption of QERA-approx does not hold for some layers with strongly correlated input dimensions.
- QERA-exact's advantage over QERA-approx diminishes as model size grows, and QPEFT calibration on downstream data (e.g., padded SST2) can fail.
Experimental setup
Datasets: GLUE SlimPajama GSM8K WikiText2 SST2 MRPC
Benchmarks: GLUE WikiText2 ARC-challenge BoolQ CommonSenseQA Winogrande MMLU BigBench-Hard GSM8K SlimPajama AlpacaEval 2.0
Metrics: accuracy perplexity win rate
Baselines: LoRA QLoRA LoftQ ZeroQuant-V2 LQER HQQ w-only quantization
Builds on: LoftQ LQER ZeroQuant-V2 QLoRA CALDERA
Closest papers by meaning
- LQER: Low-Rank Quantization Error Reconstruction for LLMs (ICML 2024 · similarity 0.92)
- QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Models (ICLR 2024 · similarity 0.90)
- Quantization Error Propagation: Revisiting Layer-Wise Post-Training Quantization (NeurIPS 2025 · similarity 0.90)
- ASER: Activation Smoothing and Error Reconstruction for Large Language Model Quantization (AAAI 2025 · similarity 0.90)
- LoTA-QAF: Lossless Ternary Adaptation for Quantization-Aware Fine-Tuning (NeurIPS 2025 · similarity 0.89)
- Exploring Post-training Quantization in LLMs from Comprehensive Study to Low Rank Compensation (AAAI 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.