Research map / Architectures and efficiency / Quantization
Towards the Limit of Network Quantization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use uniform quantization instead of k-means clustering | LeNet quantization with Huffman coding · MNIST | compression ratio: 44.58 -> 51.25 | Helped |
| Use uniform quantization instead of k-means clustering | 32-layer ResNet quantization with Huffman coding · CIFAR-10 | compression ratio: 18.25 -> 22.17 | Helped |
| Use uniform quantization instead of k-means clustering | AlexNet quantization with Huffman coding · ImageNet ILSVRC-2012 | compression ratio: 30.53 -> 40.65 | Helped |
| Use Hessian-weighted k-means instead of ordinary k-means | 32-layer ResNet quantization with fixed-length coding · CIFAR-10 | accuracy | Helped |
| Use ECSQ methods instead of k-means methods | 32-layer ResNet quantization with Huffman coding · CIFAR-10 | accuracy | Helped |
| Use Hessian computed from 1,000 rather than 50,000 samples (ablation) | Hessian-weighted k-means quantization · LeNet and 32-layer ResNet | accuracy | No clear effect |
| Replace Hessian with square roots of Adam second-moment estimates (ablation) | Hessian-weighted k-means quantization · LeNet and 32-layer ResNet | accuracy | No clear effect |
| Use Hessian-weighted means rather than ordinary means for uniform-quantizer centers (ablation) | Uniform quantization · 32-layer ResNet | accuracy | Helped |
| Use uniform quantization rather than Hessian-weighted k-means | Network quantization with fixed-length coding · 32-layer ResNet | accuracy | Hurt |
Closest papers by meaning
- Deep Neural Network Compression With Single and Multiple Level Quantization (AAAI 2018 · similarity 0.90)
- Automatic Neural Network Compression by Sparsity-Quantization Joint Learning: A Constrained Optimization-Based Approach (CVPR 2020 · similarity 0.89)
- Weighted-Entropy-Based Quantization for Deep Neural Networks (CVPR 2017 · similarity 0.89)
- Loss-aware Weight Quantization of Deep Networks (ICLR 2018 · similarity 0.88)
- And the Bit Goes Down: Revisiting the Quantization of Neural Networks (ICLR 2020 · similarity 0.88)
- Data-Free Network Compression via Parametric Non-Uniform Mixed Precision Quantization (CVPR 2022 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.