Research map / Label-efficient and robust learning / Recognition and long-tail
Enhancing Post-training Quantization Calibration through Contrastive Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Adds a contrastive-learning loss with mutual-information motivation to existing PTQ activation calibration (QDrop, PD-Quant) to optimize scaling factors and zero points for CNN quantization on ImageNet.
Problem
Post-training quantization calibration for activations typically minimizes distance metrics (MSE, cosine, KL) between FP and quantized activations without considering information loss between them, and performance collapses at very low bit widths.
Approach
Treat quantized and full-precision activations of the same sample as positive pairs and others as negatives, scored by a discriminator built from the frozen FP network's remaining layers, and add this InfoNCE-style loss to the reconstruction objective; the authors argue this maximizes a lower bound on mutual information between FP and quantized activations.
Key results
- At W2A2, CL-Calib boosts MobileNetV2 accuracy by 3.43% over the PD-Quant baseline per the text (Table 1 lists +3.53 over PD-Quant), and MNasNet-x2 by 2.31%.
- As an add-on to QDrop at 2W2A, CL-Calib improves MobileNetV2 from 10.28 to 15.48 and MNasNet-x2 from 23.59 to 28.89.
- At W4A4, gains over PD-Quant are small (e.g., +0.23 on ResNet-50 and +0.11 on ResNet-18).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add CL-Calib to PD-Quant | ResNet-18 · ImageNet, 2/2 (W/A) | accuracy: 53.08 -> 54.45 | Helped |
| Add CL-Calib to PD-Quant | ResNet-50 · ImageNet, 2/2 (W/A) | accuracy: 56.98 -> 58.30 | Helped |
| Add CL-Calib to PD-Quant | MobileNetV2 · ImageNet, 2/2 (W/A) | accuracy: 14.17 -> 17.70 | Helped |
| Add CL-Calib to PD-Quant | RegNetX-600MF · ImageNet, 2/2 (W/A) | accuracy: 40.92 -> 42.19 | Helped |
| Add CL-Calib to PD-Quant | RegNetX-3.2GF · ImageNet, 2/2 (W/A) | accuracy: 55.13 -> 56.39 | Helped |
| Add CL-Calib to PD-Quant | MNasx2 · ImageNet, 2/2 (W/A) | accuracy: 28.03 -> 30.34 | Helped |
| Add CL-Calib to PD-Quant | ResNet-18 · ImageNet, 4/2 (W/A) | accuracy: 58.65 -> 59.03 | Helped |
| Add CL-Calib to PD-Quant | ResNet-50 · ImageNet, 4/2 (W/A) | accuracy: 64.18 -> 65.12 | Helped |
| Add CL-Calib to PD-Quant | MobileNetV2 · ImageNet, 4/2 (W/A) | accuracy: 20.40 -> 22.77 | Helped |
| Add CL-Calib to PD-Quant | RegNetX-600MF · ImageNet, 4/2 (W/A) | accuracy: 51.29 -> 52.35 | Helped |
| Add CL-Calib to PD-Quant | RegNetX-3.2GF · ImageNet, 4/2 (W/A) | accuracy: 62.76 -> 63.53 | Helped |
| Add CL-Calib to PD-Quant | MNasx2 · ImageNet, 4/2 (W/A) | accuracy: 38.89 -> 40.80 | Helped |
Surprising findings
- At 4-bit weights and activations, the CL-Calib gain over PD-Quant is marginal and slightly negative on RegNetX-3.2GF (-0.17).
Limitations
- Gains at 4-bit are small and inconsistent across architectures.
- Relies on the frozen FP network's subsequent layers as the critic, which the authors note is needed because a fully learnable critic failed to optimize with limited calibration data.
- Evaluated only on CNN image classification in the main text; ViT and detection results are in the appendix.
Experimental setup
Datasets: ImageNet
Benchmarks: ImageNet
Metrics: top-1 accuracy
Baselines: QDrop PD-Quant AdaRound BRECQ LAPQ ACIQ-Mix Bit-Split AdaQuant
Builds on: QDrop PD-Quant InfoNCE BRECQ
Closest papers by meaning
- Coverage-Based Calibration for Post-Training Quantization via Weighted Maximum Coverage over Outlier Channels (NeurIPS 2026 · similarity 0.90)
- Exploring Post-training Quantization in LLMs from Comprehensive Study to Low Rank Compensation (AAAI 2024 · similarity 0.90)
- Predicting Quantization Price for Selecting PTQ Configurations Before Deployment (NeurIPS 2026 · similarity 0.88)
- ResQ: Mixed-Precision Quantization of Large Language Models with Low-Rank Residuals (ICML 2025 · similarity 0.87)
- PTQ4DiT: Post-training Quantization for Diffusion Transformers (NeurIPS 2024 · similarity 0.87)
- PTMQ: Post-training Multi-Bit Quantization of Neural Networks (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.