Research map / Architectures and efficiency / Quantization
Scalable methods for 8-bit training of neural networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace standard batch normalization with Range BN | ResNet-18 · ImageNet | error: 32.5% vs 32.4% | No clear effect |
| Replace standard batch normalization with Range BN | ResNet-56 · CIFAR-10 | error: 10.5% vs 10.7% | No clear effect |
| Remove the Range BN scale adjustment C(n) (ablation) | Range BN · simulations | accuracy | Hurt |
| Modify C(n) to nearby values (ablation) | Range BN · simulations | accuracy | Hurt |
| Use 16-bit layer-gradient copies to compute weight gradients while keeping layer-gradient propagation at 8-bit | Quantized Back-Propagation · CIFAR-10 | validation accuracy | Helped |
| Use stochastic rounding for gradient quantization | Quantized Back-Propagation · training | convergence | Helped |
| Use 8-bit rather than higher-precision quantization for the forward pass and layer-gradient propagation | Quantized Back-Propagation · ImageNet, ResNet-18 | classification accuracy | No clear effect |
Closest papers by meaning
- F8Net: Fixed-Point 8-bit Only Multiplication for Network Quantization (ICLR 2022 · similarity 0.90)
- Accurate Neural Training with 4-bit Matrix Multiplications at Standard Formats (ICLR 2023 · similarity 0.90)
- Efficient Multi-bit Quantization Network Training via Weight Bias Correction and Bit-wise Coreset Sampling (NeurIPS 2025 · similarity 0.90)
- Towards Unified INT8 Training for Convolutional Neural Network (CVPR 2020 · similarity 0.90)
- LEARNED STEP SIZE QUANTIZATION (ICLR 2020 · similarity 0.90)
- Network Quantization With Element-Wise Gradient Scaling (CVPR 2021 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.