Research map / Architectures and efficiency / Quantization
PTMQ: Post-training Multi-Bit Quantization of Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PTMQ, a post-training (no retraining) multi-bit quantization framework that uses block-wise reconstruction with a Multi-bit Feature Mixer and Group-wise Distillation Loss to support real-time bit-width switching and mixed precision for CNNs and ViTs.
Problem
Multi-bit (arbitrary bit-width) quantization normally needs costly QAT (hundreds of GPU hours), and switching bit-widths requires recalculating norm-layer statistics, which prevents real-time switching.
Approach
Extends rounding-based PTQ (BRECQ/QDrop style block reconstruction) to multi-bit settings by sharing rounding values and activation step sizes across bit-width groups, mixing quantized features from different bit groups as block inputs (MFM), and distilling high-bit group outputs into lower-bit group outputs (GD-Loss); norm layers are folded so bit switching needs no recalculation.
Key results
- On ResNet-18 at 3-bit, PTMQ reaches 64.92% top-1 vs. 64.18% for AdaRound and 64.24% for BRECQ, while using about 1.4 GPU hours versus 144 for MultiQuant.
- PTMQ achieves up to ~100x optimization speed-up over multi-bit QAT methods such as MultiQuant while remaining comparable to PTQ methods at 6-8 bit.
- On ViT/DeiT, PTMQ gains 0.3-0.5% over PTQ4ViT for ViT-S/B and 1.4-2.5% for DeiT-S/B at 4-5 bit.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use PTMQ instead of QDrop at 3-bit | ResNet-18 · ImageNet, 3-bit | top-1 accuracy: 66.26% -> 64.92% | Hurt |
| Use PTMQ instead of QDrop at 4-bit | ResNet-18 · ImageNet, 4-bit | top-1 accuracy: 69.54% -> 67.57% | Hurt |
| Use PTMQ instead of QDrop at 3-bit | ResNet-50 · ImageNet, 3-bit | top-1 accuracy: 71.07% -> 69.67% | Hurt |
| Use PTMQ instead of QDrop at 4-bit | ResNet-50 · ImageNet, 4-bit | top-1 accuracy: 74.98% -> 73.93% | Hurt |
| Use PTMQ instead of QDrop at 3-bit | RegNetX-600MF · ImageNet, 3-bit | top-1 accuracy: 64.53% -> 59.29% | Hurt |
| Use PTMQ instead of QDrop at 4-bit | RegNetX-600MF · ImageNet, 4-bit | top-1 accuracy: 70.62% -> 68.84% | Hurt |
| Use PTMQ instead of QDrop at 4-bit | MobileNet-V2 · ImageNet, 4-bit | top-1 accuracy: 67.89% -> 64.94% | Hurt |
| Use PTMQ instead of PTQ4ViT | ViT-S/224/16 · ImageNet, weights/activations 4/6 | top-1 accuracy: 71.41% -> 71.67% | Helped |
| Use PTMQ instead of PTQ4ViT | ViT-B/224/16 · ImageNet, weights/activations 4/6 | top-1 accuracy: 75.43% -> 75.00% | Hurt |
| Use PTMQ instead of PTQ4ViT | DeiT-S/224/16 · ImageNet, weights/activations 4/6 | top-1 accuracy: 74.35% -> 77.20% | Helped |
| Use PTMQ instead of PTQ4ViT | DeiT-B/224/16 · ImageNet, weights/activations 4/6 | top-1 accuracy: 77.95% -> 80.00% | Helped |
| Add mixed-precision quantization rather than uniform 4-bit quantization | PTMQ on ResNet-18 · ImageNet, ResNet-18 | top-1 accuracy: +2% | Helped |
Surprising findings
- PTMQ is slightly below BRECQ and QDrop on MobileNet-V2 (average drops of 0.76% and 1.31%), attributed to channel distribution differences.
- At 4-bit on ResNet-18, PTMQ is 0.7% below BRECQ because 3- and 4-bit share a bit group, causing intra-group competition.
Limitations
- Performance lags some PTQ baselines (BRECQ, QDrop) on MobileNet-V2 and at certain 4-bit settings.
- Bit-widths in the same group compete, reducing accuracy at some bit-widths.
- Evaluation limited to ImageNet classification.
Experimental setup
Datasets: ImageNet
Benchmarks: ImageNet
Metrics: top-1 accuracy GPU hours
Baselines: AdaRound BRECQ QDrop RobustQuant AnyPrecision MultiQuant PTQ4ViT HAWQ-V3 OMPQ LSQ
Builds on: BRECQ QDrop AdaRound MultiQuant
Closest papers by meaning
- ParetoQ: Improving Scaling Laws in Extremely Low-bit LLM Quantization (NeurIPS 2025 · similarity 0.90)
- CBQ: Cross-Block Quantization for Large Language Models (ICLR 2025 · similarity 0.90)
- Retraining-Free Model Quantization via One-Shot Weight-Coupling Learning (CVPR 2024 · similarity 0.89)
- Exploring Post-training Quantization in LLMs from Comprehensive Study to Low Rank Compensation (AAAI 2024 · similarity 0.89)
- ABQ-LLM: Arbitrary-Bit Quantized Inference Acceleration for Large Language Models (AAAI 2025 · similarity 0.89)
- AffineQuant: Affine Transformation Quantization for Large Language Models (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.