Research map / Architectures and efficiency / Quantization
Towards Accurate Network Quantization with Equivalent Smooth Regularizer
Kirill Solodskikh, Vladimir Chikin, Ruslan Aydarkhanov, Dehua Song, Irina Zhelavskaya, Jiansheng Wei
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add multistep lambda_w scheduling (1,10,100) instead of constant lambda_w (ablation) | QSin ResNet-20 · CIFAR-10 (4-bit) | accuracy (%): 91.1 -> 91.7 | Improved |
| add multistep lambda_w scheduling instead of constant lambda_w=100 (ablation) | QSin ResNet-20 · CIFAR-10 (8-bit) | accuracy (%): 91.8 -> 91.9 | Improved |
| add weight regularizer lambda_w=1 (with lambda_a=0) (ablation) | QSin ResNet-20 · CIFAR-10 (4-bit) | accuracy (%): 88.9 -> 90.0 | Improved |
| add activation regularizer lambda_a=1 (lambda_w=0) (ablation) | QSin ResNet-20 · CIFAR-10 (4-bit) | accuracy (%): 88.9 -> 89.2 | Improved |
| raise lambda_w from 1 to 100 (ablation) | QSin ResNet-20 · CIFAR-10 (4-bit) | accuracy (%): 90.2 -> 91.1 | Improved |
| regularizers on 8-bit (any lambda settings) (ablation) | QSin ResNet-20 · CIFAR-10 (8-bit) | accuracy (%): 91.6 -> 91.9 | No clear effect |
| replace MSQE regularizer with smooth QSin regularizer | ResNet-18 quantization · ImageNet (4-bit) | top-1 accuracy (%): 67.3 -> 69.7 | Improved |
| replace MSQE regularizer with QSin | MobileNet-V2 quantization · ImageNet (4-bit) | top-1 accuracy (%): 67.4 -> 68.7 | Improved |
| replace SinReQ sinusoidal regularizer with QSin | ResNet-18 quantization · ImageNet (4-bit) | top-1 accuracy (%): 64.63 -> 69.7 | Improved |
| use QSin instead of LSQ (round-free, no STE) | MobileNet-V2 quantization · ImageNet (4-bit) | top-1 accuracy (%): 68.1 -> 68.7 | Improved |
| use QSin instead of LSQ | ResNet-18 quantization · ImageNet (4-bit) | top-1 accuracy (%): 69.8 -> 69.7 | No clear effect |
| use QSin instead of MSQE for 8-bit | EDSR 4x SR · Urban100 (8-bit) | PSNR (dB): 25.9 -> 26 | Improved |
Closest papers by meaning
- Improving Low-Precision Network Quantization via Bin Regularization (ICCV 2021 · similarity 0.89)
- Gradient $\ell_1$ Regularization for Quantization Robustness (ICLR 2020 · similarity 0.89)
- Incremental Network Quantization: Towards Lossless CNNs with Low-precision Weights (ICLR 2017 · similarity 0.88)
- Training Quantized Neural Networks With a Full-Precision Auxiliary Module (CVPR 2020 · similarity 0.88)
- LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks (ECCV 2018 · similarity 0.88)
- Data-Free Quantization Through Weight Equalization and Bias Correction (ICCV 2019 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
