Research map / Architectures and efficiency / Quantization

Quantization without Tears

CVPR 2025 · Architectures and efficiency › Quantization · approach: Post-training weight quantization · 6 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Quantization without Tears (QwT), which adds a tiny closed-form-initialized linear compensation layer to each block of a PTQ-quantized network to recover accuracy in about 2 minutes.

Problem

Post-training and quantization-aware methods trade off speed against accuracy and are complex, hyperparameter-sensitive and task-specific; PTQ loses substantial accuracy at low bit-widths.

Approach

Allow the quantized network structure to include extra linear compensation modules added to each block, whose weights and biases are set in closed form via least-squares regression on 512 calibration samples to match the full-precision block outputs, with no hyperparameters and no backpropagation required for the basic version.

Key results

• On ImageNet with 4-bit RepQ-ViT on DeiT-T/Swin-T/ViT-B, QwT adds about 2.6 points on average and up to 5 points; ViT-B 4/4 goes from 68.5 to 76.3 top-1 (78.5 with one epoch of finetuning).
• On LLaMA3-8B with GPTQ INT4, QwT improves C4 perplexity from 9.44 to 9.38 and the eight-task QA average from 64.90 to 65.18.
• For DiT-XL/2 W8A8 and W4A8, QwT improves FID by 0.10 and 0.69 over Q-DiT.

Measured effects reported in the paper

Surprising findings

• Applying the closed-form initialization directly to QAT models (e.g., Q-ViT) significantly degrades accuracy, so QAT models need zero initialization and fine-tuning instead.

Limitations

• Closed-form compensation is not effective for QAT models and requires fine-tuning there.
• Roughly 2-4% extra latency and some model-size increase compared to plain PTQ; gains on CLIP text encoder and LLM are modest.
• Initialization is skipped for a small subset (<5%) of modules with low R^2, so the method is not guaranteed to help every block.

Experimental setup

Datasets: ImageNet COCO 2017 WikiText2 C4 SIQA HellaSwag PIQA WinoGrande ARC BoolQ OpenBookQA MMLU

Benchmarks: ImageNet COCO 2017 WikiText2 C4 MMLU SIQA HellaSwag PIQA WinoGrande ARC BoolQ OpenBookQA

Metrics: Top-1 accuracy AP box AP mask FID IS perplexity QA average accuracy latency model size

Baselines: RepQ-ViT PTQ4ViT Percentile IGQ-ViT CL-Calib MinMax GPTQ Q-DiT Q-ViT

Builds on: RepQ-ViT GPTQ Percentile Q-DiT Q-ViT

Closest papers by meaning

• Focused Quantization for Sparse CNNs (NeurIPS 2019 · similarity 0.90)
• And the Bit Goes Down: Revisiting the Quantization of Neural Networks (ICLR 2020 · similarity 0.90)
• Bit-Shrinking: Limiting Instantaneous Sharpness for Improving Post-Training Quantization (CVPR 2023 · similarity 0.89)
• Deep Neural Network Quantization via Layer-Wise Optimization Using Limited Training Data (AAAI 2019 · similarity 0.89)
• Effective Interplay between Sparsity and Quantization: From Theory to Practice (ICLR 2025 · similarity 0.88)
• Network Memory Footprint Compression Through Jointly Learnable Codebooks and Mappings (ICLR 2024 · similarity 0.88)