Research map / Architectures and efficiency / Quantization
Weighted-Entropy-Based Quantization for Deep Neural Networks
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use weighted-entropy quantization with 2-bit weights and 3-bit activations instead of the cited 8-bit quantization | AlexNet · ImageNet | top-5 accuracy: 76.6% vs. 75.49% | No clear effect |
| Use 2-bit weights and 3-bit activations instead of 1-bit weights and 4-bit activations | AlexNet · ImageNet | top-1 accuracy: 53.0 vs. 53.7; 0.69% | Helped |
| Add weighted-entropy quantization to Deep Compression | AlexNet · ImageNet | top-1 accuracy: 57.2 -> 56.3; 0.9% | Hurt |
| Use a convex layer-wise bitwidth profile rather than a monotonically increasing profile (ablation) | Weighted entropy quantization · AlexNet, ImageNet | top-1 accuracy: 50.35 -> 54.33 | Helped |
| Use a convex rather than monotonically decreasing layer-wise bitwidth profile (ablation) | Weighted entropy quantization · AlexNet, ImageNet | top-5 accuracy: 77.59 -> 78.20 | Helped |
| Quantize LSTM weights to 1 bit rather than full precision | LSTM Large · Penn Tree Bank valid | word-level perplexity: 82.77 -> 92.20 | Hurt |
| Quantize LSTM weights to 4 bits rather than full precision | LSTM Large · Penn Tree Bank test | word-level perplexity: 78.63 -> 78.09 | No clear effect |
| Use 1-bit rather than 4-bit weights | LSTM Large · Penn Tree Bank test | word-level perplexity: 78.09 -> 88.48 | Hurt |
| Use monotonically increasing bitwidths across layers rather than monotonically decreasing bitwidths (ablation) | Weighted entropy quantization · AlexNet, ImageNet | top-1 accuracy: 53.79 -> 50.35 | Hurt |
Closest papers by meaning
- Efficient Quantization for Neural Networks with Binary Weights and Low Bitwidth Activations (AAAI 2019 · similarity 0.90)
- Quantization Networks (CVPR 2019 · similarity 0.90)
- Loss-aware Weight Quantization of Deep Networks (ICLR 2018 · similarity 0.89)
- Analysis of Quantized Models (ICLR 2019 · similarity 0.89)
- Towards the Limit of Network Quantization (ICLR 2017 · similarity 0.89)
- Explicit Loss-Error-Aware Quantization for Low-Bit Deep Neural Networks (CVPR 2018 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.