Research map / Architectures and efficiency / Quantization

Quantization Error Propagation: Revisiting Layer-Wise Post-Training Quantization

NeurIPS 2025 · Architectures and efficiency › Quantization · approach: Post-training quantization of LLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Quantization Error Propagation (QEP), a lightweight correction to layer-wise PTQ that uses quantized-input Hessians plus a tunable propagation strength alpha to compensate accumulated quantization errors, improving GPTQ, AWQ, QuIP, and RTN especially at INT2.

Problem

Layer-wise post-training quantization of LLMs optimizes each layer independently, so quantization errors accumulate and grow across layers, severely degrading accuracy in low-bit (especially 2-bit) settings.

Approach

Reformulate the layer-wise objective to match full-precision outputs computed from full-precision inputs while using quantized inputs, shift the weights by a closed-form correction term W·δ·X̂ᵀ Ĥ⁻¹ so the existing Hessian-based quantizers can be reused, and add a tunable scalar alpha in [0,1] that acts as a regularizer against overfitting.

Key results

• At INT2 on WikiText2, QEP+QuIP reduces Llama-2-7B perplexity from 65.593 to 11.972 and Llama-3-8B from 70.518 to 27.326.
• At INT3 on WikiText2, QEP+GPTQ lowers Llama-2-7B perplexity from 10.881 to 7.898 and QEP+RTN lowers Llama-3-8B from 2276.227 to 86.430.
• QEP-enhanced QuIP achieves the best perplexity among tested layer-wise PTQ methods at INT2 for several models.

Measured effects reported in the paper

Surprising findings

• QEP+GPTQ at INT2 on Llama-3-8B (282245) and QEP+AWQ at INT2 on Llama-2-7B/Llama-3-8B are worse in perplexity than the baseline, showing QEP does not uniformly help.
• GPTQ with default calibration gave a severe Llama-3-8B INT4 zero-shot drop (0.4812), while QEP+GPTQ recovers it to 0.7531 in the reported table.

Limitations

• Relies on a small calibration set, so performance is sensitive to calibration data quality.
• Introduces a per-layer propagation-strength hyperparameter alpha that is fixed at 1/2 by default rather than learned automatically.
• Theoretical guarantees rely on first-order and Lipschitz assumptions, and gains are not uniform across all method/bit combinations.

Experimental setup

Datasets: C4 Pile WikiText2

Benchmarks: WikiText2 PTB C4 ARC-Easy PIQA StoryCloze

Metrics: perplexity zero-shot accuracy

Baselines: RTN GPTQ AWQ QuIP

Builds on: GPTQ AWQ QuIP

Closest papers by meaning

• CBQ: Cross-Block Quantization for Large Language Models (ICLR 2025 · similarity 0.92)
• ASER: Activation Smoothing and Error Reconstruction for Large Language Model Quantization (AAAI 2025 · similarity 0.91)
• QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Models (ICLR 2024 · similarity 0.91)
• Exploring Post-training Quantization in LLMs from Comprehensive Study to Low Rank Compensation (AAAI 2024 · similarity 0.91)
• Norm Tweaking: High-Performance Low-Bit Quantization of Large Language Models (AAAI 2024 · similarity 0.91)
• LeanQuant: Accurate and Scalable Large Language Model Quantization with Loss-error-aware Grid (ICLR 2025 · similarity 0.91)