Research map / Architectures and efficiency / Quantization
Training Quantized Nets: A Deeper Understanding
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use BinaryConnect with a full-precision weight buffer instead of stochastic rounding | binarized-weight training · CIFAR-10 ResNet-56 | top-1 test error: 26.49 -> 8.83 | Helped |
| Use BinaryConnect with a full-precision weight buffer instead of stochastic rounding | binarized-weight training · ImageNet ResNet-18 | top-1 test error: 88.86 -> 52.11 | Helped |
| Use stochastic rounding instead of deterministic rounding | binarized-weight training · CIFAR-10 ResNet-56 | top-1 test error: 33.56 -> 26.49 | Helped |
| Increase the minibatch size for SR-ADAM (ablation) | SR-ADAM · CIFAR-10 ResNet-56 | top-1 test error: 26.49 -> 19.84 | Helped |
| Increase the minibatch size for SR-ADAM (ablation) | SR-ADAM · ImageNet ResNet-18 | top-1 test error: 88.86 -> 77.68 | Helped |
| Increase the minibatch size for SR-ADAM (ablation) | SR-ADAM · CIFAR-10 VGG-9 | top-1 test error: 23.33 -> 16.95 | Helped |
| Increase the minibatch size for SR-ADAM (ablation) | SR-ADAM · CIFAR-10 VGG-BC | top-1 test error: 20.56 -> 16.77 | Helped |
| Increase the minibatch size for SR-ADAM (ablation) | SR-ADAM · CIFAR-10 WRN-56-2 | top-1 test error: 21.58 -> 16.04 | Helped |
| Increase the minibatch size for SR-ADAM (ablation) | SR-ADAM · CIFAR-100 ResNet-56 | top-1 test error: 58.06 -> 50.79 | Helped |
| Reduce the learning rate in stochastic rounding | SR · non-convex optimization | concentration on local minimizers | No clear effect |
| Maintain a full-precision buffer while computing gradients on quantized weights | BinaryConnect · non-convex optimization | concentration on local minimizers | Helped |
| Use larger minibatches for stochastic rounding (ablation) | SR-ADAM · CIFAR-10 ResNet-56 | test error | Helped |
Closest papers by meaning
- Analysis of Quantized Models (ICLR 2019 · similarity 0.91)
- Training Quantized Neural Networks to Global Optimality via Semidefinite Programming (ICML 2021 · similarity 0.90)
- Towards Efficient Verification of Quantized Neural Networks (AAAI 2024 · similarity 0.89)
- Don’t Waste Your Bits! Squeeze Activations and Gradients for Deep Neural Networks via TinyScript (ICML 2020 · similarity 0.89)
- Network Quantization With Element-Wise Gradient Scaling (CVPR 2021 · similarity 0.89)
- A Mean Field Theory of Quantized Deep Networks: The Quantization-Depth Trade-Off (NeurIPS 2019 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.