Research map / Architectures and efficiency / Quantization
Deep Learning With Low Precision by Half-Wave Gaussian Quantization
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace XNOR-Net with HWGQ-Net using 1-bit weights and 2-bit activations | AlexNet · ImageNet (ILSVRC2012) | Top-1 accuracy: 44.2 -> 52.7 | Helped |
| Replace DOREFA-Net with HWGQ-Net using 1-bit weights and 2-bit activations | AlexNet · ImageNet (ILSVRC2012) | Top-1 accuracy: 47.7 -> 52.7 | Helped |
| Use clipped ReLU instead of vanilla ReLU as the backward approximation (ablation) | HWGQ-Net with ResNet-18 · ImageNet (ILSVRC2012) | Top-1 accuracy: 33.0 -> 54.5 | Helped |
| Use clipped ReLU instead of vanilla ReLU as the backward approximation (ablation) | HWGQ-Net with AlexNet · ImageNet (ILSVRC2012) | Top-1 accuracy: 46.8 -> 48.6 | Helped |
| Use log-tailed ReLU instead of clipped ReLU as the backward approximation (ablation) | HWGQ-Net with AlexNet · ImageNet (ILSVRC2012) | Top-1 accuracy: 48.6 -> 49.0 | Helped |
| Use log-tailed ReLU instead of clipped ReLU as the backward approximation (ablation) | HWGQ-Net with ResNet-18 · ImageNet (ILSVRC2012) | Top-1 accuracy: 54.5 -> 53.5 | Hurt |
| Train quantized activations from scratch instead of quantizing a pretrained binary-weight network without optimization (ablation) | HWGQ-Net with AlexNet · ImageNet (ILSVRC2012) | Top-1 accuracy: 30.0 -> 48.6 | Helped |
| Increase non-uniform positive activation levels from 2 to 7 (ablation) | HWGQ-Net with AlexNet · ImageNet (ILSVRC2012) | Top-1 accuracy: 48.6 -> 52.4 | Helped |
| Increase non-uniform positive activation levels from 7 to 15 (ablation) | HWGQ-Net with AlexNet · ImageNet (ILSVRC2012) | Top-1 accuracy: 52.4 -> 52.6 | No clear effect |
| Use 3 uniform levels instead of 2 non-uniform positive levels plus zero (ablation) | HWGQ-Net with AlexNet · ImageNet (ILSVRC2012) | Top-1 accuracy: 48.6 -> 50.5 | Helped |
| Replace full-precision activations with HWGQ activations (ablation) | AlexNet with binary weights · ImageNet (ILSVRC2012) | Top-1 accuracy: 52.4 -> 46.8 | Hurt |
| Replace binary sign activations with HWGQ activations (ablation) | AlexNet with binary weights · ImageNet (ILSVRC2012) | Top-1 accuracy: 39.5 -> 46.8 | Helped |
Closest papers by meaning
- SYQ: Learning Symmetric Quantization for Efficient Deep Neural Networks (CVPR 2018 · similarity 0.87)
- Don’t Waste Your Bits! Squeeze Activations and Gradients for Deep Neural Networks via TinyScript (ICML 2020 · similarity 0.87)
- Quantization Networks (CVPR 2019 · similarity 0.86)
- Network Quantization With Element-Wise Gradient Scaling (CVPR 2021 · similarity 0.86)
- Linear Symmetric Quantization of Neural Networks for Low-precision Integer Hardware (ICLR 2020 · similarity 0.86)
- LEARNED STEP SIZE QUANTIZATION (ICLR 2020 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.