Research map / Architectures and efficiency / Quantization
Cluster-Promoting Quantization With Bit-Drop for Minimizing Network Quantization Loss
Jung Hyun Lee, Jihun Yun, Sung Ju Hwang, Eunho Yang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add multi-class STE and probabilistic parametrization (CPQ) in place of RQ (ablation) | RQ · CIFAR-10 VGG-7 (W/A 4/4 bit) | test error (%): 8.43 -> 7.15 | Improved |
| add multi-class STE and probabilistic parametrization (CPQ) in place of RQ (ablation) | RQ · MNIST LeNet-5 (W/A 3/3 bit) | test error (%): 0.69 -> 0.67 | Improved |
| add DropBits to CPQ (ablation) | CPQ · CIFAR-10 VGG-7 (W/A 4/4 bit) | test error (%): 7.15 -> 6.85 | Improved |
| add DropBits to CPQ (ablation) | CPQ · MNIST LeNet-5 (W/A 3/3 bit) | test error (%): 0.67 -> 0.58 | Improved |
| add DropBits to CPQ (ablation) | CPQ · CIFAR-10 VGG-7 (W/A 2/2 bit) | test error (%): 7.68 -> 7.51 | Improved |
| add temperature annealing of the Gumbel-Softmax trick (ablation) | RQ · CIFAR-10 VGG-7 (W/A 3/3 bit) | test error (%): 9.56 -> 10.78 | Hurt |
| replace Gumbel-Softmax backward pass with multi-class STE (ablation) | RQ · CIFAR-10 VGG-7 (W/A 3/3 bit) | test error (%): 9.56 -> 8.44 | Improved |
| replace Gumbel-Softmax forward pass with mode selection plus DropBits (ablation) | Gumbel-Softmax + multi-class STE · CIFAR-10 VGG-7 (W/A 3/3 bit) | test error (%): 8.44 -> 6.94 | Improved |
| learn per-layer bit-width with the DropBits regularizer instead of fixing the same bit-widths from scratch (ablation) | CPQ + DropBits · ImageNet ResNet-18 (W/A 3/3 bit) | top-1 error (%): 41.01 -> 40.30 | Improved |
| learn per-layer bit-width with the DropBits regularizer instead of fixing the same bit-widths from scratch (ablation) | CPQ + DropBits · VGG-7 CIFAR-10 (W/A 2/2 bit, lambda variant) | test error (%): 9.62 -> 7.55 | Improved |
| learn per-layer bit-width with the DropBits regularizer instead of fixing the same bit-widths from scratch (ablation) | CPQ + DropBits · MNIST LeNet-5 (W/A 3/3 bit) | test error (%): 0.65 -> 0.55 | Improved |
| learn heterogeneous bit-widths with regularization instead of uniform quantization (ablation) | CPQ + DropBits · VGG-7 CIFAR-10 and ResNet-18 ImageNet (W/A 4/4 bit) | test error (%): 6.77 -> 6.65 (VGG-7); 33.20 -> 34.30 (ResNet-18) | Mixed |
Closest papers by meaning
- LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks (ECCV 2018 · similarity 0.89)
- QBB: Quantization with Binary Bases for LLMs (NeurIPS 2024 · similarity 0.89)
- Towards the Limit of Network Quantization (ICLR 2017 · similarity 0.89)
- Improving Low-Precision Network Quantization via Bin Regularization (ICCV 2021 · similarity 0.89)
- Bit-Shrinking: Limiting Instantaneous Sharpness for Improving Post-Training Quantization (CVPR 2023 · similarity 0.88)
- Pushing the Limit of Post-Training Quantization (TPAMI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
