Research map / Architectures and efficiency / Quantization

Predicting Quantization Price for Selecting PTQ Configurations Before Deployment

NeurIPS 2026 · Architectures and efficiency › Quantization · approach: Post-training quantization of ViTs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Formulates weight-space PTQ as pre-deployment configuration selection and prices candidate layer-output error covariances via a forward-KL quadratic term, yielding a trace-based budgeted selector.

Problem

Weight-space PTQ must choose formats, granularities, quantizer families, transformations, and bits before the completed quantized model reveals its output-distribution drift; existing proxies score only parts of this decision.

Approach

Treat each admissible layer configuration as an error generator with deployment cost; forward KL at the full-precision reference cancels the first-order term, so the drift is priced by a quadratic form of candidate-induced layer-output error covariance weighted by reference output curvature. A trace reduction gives a calibration-time price table used in a budgeted selector.

Key results

• Realized layer-output price correlates with measured KL drift at rho=0.9470 on OPT-125M and 0.9249 on Qwen3-0.6B.
• Implemented trace price keeps near-equal alignment with KL drift (rho=0.9483 OPT-125M, 0.9438 Qwen3-0.6B).
• On Llama-3.2-1B bit allocation at 3.0 average bits, the price-guided selector reaches avg 51.24 and PPL 12.22 in 579 s versus HIGGS 12.50 PPL / 813 s and AMQ 11635 s with PPL 11.05.

Surprising findings

• AMQ achieves the lowest WikiText2 perplexity (11.05) in bit allocation, beating the price-guided selector, though the selector is far faster and has higher average task accuracy.
• Random granularity selection yields PPL 23313 on Llama-3.2-1B, far worse than fixed group-128.

Limitations

• Price is a local quadratic approximation valid only in low-bit-nonlocality, moment-stable, curvature-aligned regimes; the paper states that other regimes are outside scope.
• Trace reduction relies on approximately isotropic candidate covariance; diagonal and imatrix-style proxies drop factors that may vary across candidates.
• Evaluation is limited to small models (0.125B to 1B) and weight-only PTQ; activation quantization is excluded from the main selector.

Experimental setup

Datasets: WikiText2

Benchmarks: WikiText2 BoolQ TruthfulQA PIQA WinoGrande WiC

Metrics: perplexity accuracy average accuracy selection time correlation with KL drift

Baselines: HIGGS AMQ CALM-CKA Fixed Group-128 Random Selection

Builds on: GPTQ HAWQ-V3 HIGGS AMQ QuaRot

Closest papers by meaning

• Coverage-Based Calibration for Post-Training Quantization via Weighted Maximum Coverage over Outlier Channels (NeurIPS 2026 · similarity 0.88)
• Exploring Post-training Quantization in LLMs from Comprehensive Study to Low Rank Compensation (AAAI 2024 · similarity 0.88)
• Enhancing Post-training Quantization Calibration through Contrastive Learning (CVPR 2024 · similarity 0.88)
• Quantization Error Propagation: Revisiting Layer-Wise Post-Training Quantization (NeurIPS 2025 · similarity 0.87)
• Q-Palette: Fractional-Bit Quantizers Toward Optimal Bit Allocation for Efficient LLM Deployment (NeurIPS 2025 · similarity 0.87)
• GSQ: Highly-Accurate Low-Precision Scalar Quantization for LLMs via Gumbel-Softmax Sampling (NeurIPS 2026 · similarity 0.86)