Research map / Architectures and efficiency / Quantization
Efficient Quantization for Neural Networks with Binary Weights and Low Bitwidth Activations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| increase weight binarization levels from m=1 to m=3 (ablation) | Ours · CIFAR-10 | top-1 error: 7.60% -> 6.42% | Helped |
| use refined backward gradient (Eq. 10) instead of XNOR-Net style gradient (BWN-net vs Ours m=1) (ablation) | binary weights VGG-variant · CIFAR-10 | top-1 error: 7.84% -> 7.60% | Helped |
| use refined backward gradient compared with Sketch (ablation) | ResNet-18 binary weights · ImageNet | top-1 error: 32.7% -> 32.5% | Helped |
| increase activation bitwidth from 2 to 5 bits (ablation) | binary-weight ResNet-50 · ImageNet | top-1 error: 27.8% -> 23.4% | Helped |
| replace ReLU with CReLU under quantization (ablation) | quantized ResNet-20/32 · CIFAR-10 | top-1 error | Helped |
| replace ReLU with CReLU without quantization (ablation) | ResNet-20/32 · CIFAR-10 | top-1 error | No clear effect |
| use learnable clamping (CReLU) instead of fixed clamping values (ablation) | quantized ResNet · CIFAR-10 | top-1 error | Helped |
| use L2 instead of L1 regularization on clamping parameter (ablation) | CReLU · CIFAR-10 | accuracy | Helped |
| use binary weights with 2-bit activations vs HWGQ-Net | AlexNet · ImageNet | top-1 error: 47.3% -> 46.2% | Helped |
| use binary weights with 2-bit activations vs HWGQ-Net | ResNet-18 · ImageNet | top-1 error: 40.4% -> 38.4% | Helped |
Closest papers by meaning
- Quantization Networks (CVPR 2019 · similarity 0.91)
- Weighted-Entropy-Based Quantization for Deep Neural Networks (CVPR 2017 · similarity 0.90)
- Accurate Neural Training with 4-bit Matrix Multiplications at Standard Formats (ICLR 2023 · similarity 0.89)
- Incremental Network Quantization: Towards Lossless CNNs with Low-precision Weights (ICLR 2017 · similarity 0.89)
- Searching for Low-Bit Weights in Quantized Neural Networks (NeurIPS 2020 · similarity 0.89)
- QBB: Quantization with Binary Bases for LLMs (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.