Research map / Architectures and efficiency / Quantization
Accumulation Bit-Width Scaling For Ultra-Low Precision Training Of Deep Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Set accumulation precision using the VRR analysis | CIFAR-10 ResNet 32 · CIFAR-10 | test accuracy degradation vs. full-precision accumulation: within 0.5% | No clear effect |
| Set accumulation precision using the VRR analysis | ImageNet ResNet 18 · ImageNet | test accuracy degradation vs. full-precision accumulation: within 0.5% | No clear effect |
| Set accumulation precision using the VRR analysis | ImageNet AlexNet · ImageNet | test accuracy degradation vs. full-precision accumulation: within 0.5% | No clear effect |
| Reduce accumulation precision below the VRR prediction (ablation) | CIFAR-10 ResNet 32 and ImageNet ResNet 18 · CIFAR-10 and ImageNet | test accuracy: PP<0 | Hurt |
| Reduce accumulation precision by one bit from the prediction (ablation) | ImageNet AlexNet · ImageNet | test accuracy degradation vs. baseline: strictly > 0.5% | Hurt |
| Reduce accumulation precision further below the prediction (ablation) | CIFAR-10 ResNet 32, ImageNet ResNet 18, and ImageNet AlexNet · CIFAR-10 and ImageNet | test accuracy | Hurt |
| Use chunk-based instead of normal accumulation | VRR-predicted precision assignments · CIFAR-10 ResNet 32 and ImageNet ResNet 18 and AlexNet | required accumulation mantissa precision: 1 to 6 bits | Helped |
| Use chunk-based accumulation (ablation) | CIFAR-10 ResNet 32 and ImageNet ResNet 18 and AlexNet · CIFAR-10 and ImageNet | test accuracy sensitivity to precision perturbation | Hurt |
Closest papers by meaning
- Towards Cheaper Inference in Deep Networks with Lower Bit-Width Accumulators (ICLR 2024 · similarity 0.89)
- Not All Bits have Equal Value: Heterogeneous Precisions via Trainable Noise (NeurIPS 2022 · similarity 0.89)
- Mixed Precision Training (ICLR 2018 · similarity 0.89)
- Training Deep Neural Networks with 8-bit Floating Point Numbers (NeurIPS 2018 · similarity 0.87)
- Scaling Laws for Floating–Point Quantization Training (ICML 2025 · similarity 0.85)
- AMPA: Adaptive Mixed Precision Allocation for Low-Bit Integer Training (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.