Research map / Architectures and efficiency / Quantization
Retraining-Free Model Quantization via One-Shot Weight-Coupling Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a one-shot weight-coupling training-then-search paradigm for retraining-free mixed-precision quantization, using bit-width scheduling, information distortion mitigation, and an inference-only bidirectional greedy search.
Problem
Mixed-precision quantization requires a costly retraining stage after bit-width search, and weight-sharing across many bit-width configurations suffers from bit-width interference that degrades training and accuracy at high compression.
Approach
Couple all bit-width configurations in one shared weight set, periodically freeze the most unstable (smallest) bit-width of layers chosen by rounding-error proxies, align small-bit feature outputs to high-bit ones via a rectified distance loss, and then search policies greedily with an inference-only evaluation so no retraining is needed.
Key results
- Under 4-bit MP on ResNet, the method reaches 71.0% Top-1 at 31.6G BitOPs with zero retraining epochs, versus LIMPQ's 70.8% at 35.0G with 90 epochs.
- On MobileNetV2 4MP/4MP, the method reaches 70.7% Top-1 at 5.5G BitOPs with no retraining, versus QBR's 70.4% at 5.4G with 90 epochs.
- Ablation on the weight-sharing model at 4-bit: baseline 68.3%, with dynamic bit schedule 69.1%, with both techniques 69.5%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use one-shot mixed-precision quantization instead of SEAM | ResNet18 · ImageNet | Top-1 accuracy: 70.8 -> 71.0 | Helped |
| use one-shot mixed-precision quantization instead of QBR | MobileNetV2 · ImageNet, 3-bit configuration | Top-1 accuracy: 67.4 -> 67.8 | Helped |
| use one-shot mixed-precision quantization instead of QBitOPT | EfficientNetLite-B0 · ImageNet, 4MP/4MP | Top-1 accuracy: 73.3 -> 73.2 | Hurt |
| remove the 2-bit option from weight-sharing quantization (ablation) | MobileNetV2 · ImageNet, 6-bit weight-sharing model | Top-1 accuracy: 69.2 -> 70.4 | Helped |
| remove the 2-bit option from weight-sharing quantization (ablation) | MobileNetV2 · ImageNet, 4-bit weight-sharing model | Top-1 accuracy: 68.1 -> 69.1 | Helped |
| add dynamic bit scheduling (ablation) | ResNet weight-sharing model · ImageNet | Top-1 accuracy: 68.3 -> 69.1 | Helped |
| add information distortion mitigation to dynamic bit scheduling (ablation) | ResNet weight-sharing model · ImageNet | Top-1 accuracy: 69.1 -> 69.5 | Helped |
| add dynamic bit scheduling and information distortion mitigation (ablation) | ResNet weight-sharing model · ImageNet | Top-1 accuracy: 68.3 -> 69.5 | Helped |
Surprising findings
- Adding a very low bit-width (2-bit) into the weight-sharing training destabilizes higher-bit convergence, so simply dropping ultra-low bit-widths was the prior workaround.
Limitations
- Transfer learning results show accuracy drops of up to 2.0% at 3-bit on Pets, so low-bit transfer is not lossless.
- Evaluated only on three lightweight CNN models and image classification datasets.
Experimental setup
Datasets: ImageNet CIFAR100 Pets
Benchmarks: ImageNet CIFAR100 Pets
Metrics: Top-1 accuracy BitOPs retrain cost (epochs)
Baselines: PACT LSQ EWGS EdMIPS GMPQ DNAS FracBits LIMPQ SEAM MPQCO QBR QBitOPT
Builds on: LSQ LIMPQ ABN AdaBits HAQ
Closest papers by meaning
- PTMQ: Post-training Multi-Bit Quantization of Neural Networks (AAAI 2024 · similarity 0.89)
- Learning from Loss Landscape: Generalizable Mixed-Precision Quantization via Adaptive Sharpness-Aware Gradient Aligning (ICML 2025 · similarity 0.89)
- Efficient Multi-bit Quantization Network Training via Weight Bias Correction and Bit-wise Coreset Sampling (NeurIPS 2025 · similarity 0.87)
- SLiM: One-shot Quantization and Sparsity with Low-rank Approximation for LLM Weight Compression (ICML 2025 · similarity 0.87)
- SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression (ICLR 2024 · similarity 0.87)
- ScaleBITS: Scalable Bitwidth Search for Hardware-Aligned Mixed-Precision LLMs (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.