Research map / Optimization / Optimizers
A Statistical Framework for Low-bitwidth Training of Deep Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace per-tensor gradient quantizer with per-sample quantizer (PSQ) (ablation) | FQT gradient quantization · ImageNet ResNet50, 5-bit gradient | top-1 accuracy (%): 74.62 -> 76.30 | Helped |
| replace per-tensor gradient quantizer with block Householder quantizer (BHQ) (ablation) | FQT gradient quantization · ImageNet ResNet50, 5-bit gradient | top-1 accuracy (%): 74.62 -> 76.83 | Helped |
| replace PTQ with BHQ at 4-bit gradient (ablation) | FQT on ResNet50 · ImageNet ResNet50, 4-bit gradient | top-1 accuracy (%): diverge -> 74.75 | Helped |
| replace PTQ with BHQ at 4-bit gradient (ablation) | FQT on ResNet18 · ImageNet ResNet18, 4-bit gradient | top-1 accuracy (%): 68.70 -> 69.48 | Helped |
| replace PTQ with PSQ at 8-bit gradient (ablation) | FQT on ResNet18 · ImageNet ResNet18, 8-bit gradient | top-1 accuracy (%): 71.24 -> 70.92 | Hurt |
| replace PTQ with PSQ at 8-bit gradient (ablation) | FQT on ResNet50 · ImageNet ResNet50, 8-bit gradient | top-1 accuracy (%): 76.40 -> 77.40 | Helped |
| replace PTQ with PSQ then BHQ (ablation) | gradient quantizer · CIFAR10 conv_3_5_2 gradient | quantizer variance: 1.2e-3 -> 8.1e-5 -> 1.4e-5 | Helped |
| reduce gradient bitwidth by one bit (ablation) | per-tensor quantizer · CIFAR10 | quantization variance: about 4x per bit | Hurt |
| replace PTQ with BHQ at 5-bit gradient (ablation) | Transformer FQT · IWSLT14 En-De | validation BLEU | Helped |
Closest papers by meaning
- Compute-Optimal Quantization-Aware Training (ICLR 2026 · similarity 0.92)
- Accurate Neural Training with 4-bit Matrix Multiplications at Standard Formats (ICLR 2023 · similarity 0.90)
- Analysis of Quantized Models (ICLR 2019 · similarity 0.90)
- Network Quantization With Element-Wise Gradient Scaling (CVPR 2021 · similarity 0.90)
- Post-Training Sparsity-Aware Quantization (NeurIPS 2021 · similarity 0.89)
- Efficient Multi-bit Quantization Network Training via Weight Bias Correction and Bit-wise Coreset Sampling (NeurIPS 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.