Research map / Architectures and efficiency / Quantization
SQuant: On-the-Fly Data-Free Quantization via Diagonal Hessian Approximation
Cong Guo, Yuxian Qiu, Jingwen Leng, Xiaotian Gao, Chen Zhang, Yunxin Liu, Fan Yang, Yuhao Zhu, Minyi Guo
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SQuant instead of the strongest listed baseline | ResNet18, 4-bit weight and activation quantization · ImageNet | Top-1: 60.60 -> 66.14 | Improved |
| Use SQuant instead of the strongest listed baseline | ResNet50, 4-bit weight and activation quantization · ImageNet | Top-1: 55.65 -> 70.80 | Improved |
| Use SQuant instead of the strongest listed baseline | InceptionV3, 8-bit weight and activation quantization · ImageNet | Top-1: 78.78 -> 78.79 | No clear effect |
| Use SQuant instead of the strongest listed baseline | SqueezeNext, 4-bit weight and activation quantization · ImageNet | Top-1: 28.93 -> 43.45 | Improved |
| Use SQuant instead of the strongest listed baseline | ShuffleNet, 8-bit weight and activation quantization · ImageNet | Top-1: 64.34 -> 64.68 | Improved |
| Add output-channel and kernel-wise optimization to element-wise rounding (ablation) | SQuant on ResNet18 · ImageNet (weight-only, 3-bit) | Top-1: 2.05 -> 60.78 | Improved |
| Add output-channel and kernel-wise optimization to element-wise rounding (ablation) | SQuant on ResNet18 · ImageNet (weight-only, 4-bit) | Top-1: 48.15 -> 69.75 | Improved |
| Add output-channel optimization to element-wise rounding (ablation) | SQuant on ResNet18 · ImageNet (weight-only, 4-bit) | Top-1: 48.15 -> 67.14 | Improved |
| Add kernel-wise optimization to element-wise rounding (ablation) | SQuant on ResNet18 · ImageNet (weight-only, 4-bit) | Top-1: 48.15 -> 68.07 | Improved |
| Add output-channel optimization to element-wise and kernel-wise optimization (ablation) | SQuant on ResNet18 · ImageNet (weight-only, 4-bit) | Top-1: 68.07 -> 69.75 | Improved |
Later papers that compared with this method
4 later accepted papers report SQuant as a baseline in their results tables; 4 of them report a better number on most of the results they share with it.
- PowerQuant: Automorphism Search for Non-Uniform Quantization (ICLR 2023 · better on 5 of 5 shared results)
- REx: Data-Free Residual Quantization Error Expansion (NeurIPS 2023 · better on 5 of 5 shared results)
- Diverse Sample Generation: Pushing the Limit of Generative Data-Free Quantization (TPAMI 2023 · better on 4 of 4 shared results)
- Unified Data-Free Compression: Pruning and Quantization without Fine-Tuning (ICCV 2023 · better on 3 of 3 shared results)
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- A Statistical Framework for Low-bitwidth Training of Deep Neural Networks (NeurIPS 2020 · similarity 0.87)
- HAWQ: Hessian AWare Quantization of Neural Networks With Mixed-Precision (ICCV 2019 · similarity 0.87)
- Towards the Limit of Network Quantization (ICLR 2017 · similarity 0.87)
- Adaptive Quantization of Neural Networks (ICLR 2018 · similarity 0.87)
- Explicit Loss-Error-Aware Quantization for Low-Bit Deep Neural Networks (CVPR 2018 · similarity 0.87)
- QBB: Quantization with Binary Bases for LLMs (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
