Research map / Architectures and efficiency / Quantization
Value-aware Quantization for Training and Inference of Neural Networks
Eunhyeok Park, Sungjoo Yoo, Peter Vajda
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add 2% large activations kept at high precision to 2-bit V-Quant (ablation) | V-Quant ResNet-50 · ImageNet val (training with quantized activations) | top-1 accuracy: 65.754 -> 75.638 | Improved |
| add 1% large activations to 1-bit V-Quant (ablation) | V-Quant ResNet-50 · ImageNet val (training with quantized activations) | top-1 accuracy: 5.302 -> 74.510 | Improved |
| add 2% large activations to 3-bit V-Quant (ablation) | V-Quant ResNet-50 · ImageNet val (training with quantized activations) | top-1 accuracy: 75.486 -> 75.920 | Improved |
| add 2% large activations to 3-bit RV-Quant (ablation) | RV-Quant ResNet-50 · ImageNet val (training with quantized activations) | top-1 accuracy: 75.156 -> 75.932 | Improved |
| add 1% large activations to 2-bit RV-Quant (ablation) | RV-Quant ResNet-50 · ImageNet val (training with quantized activations) | top-1 accuracy: 35.518 -> 75.338 | Improved |
| use RV-Quant mask sharing instead of V-Quant with separate bit mask (ablation) | 3-bit 2% ResNet-50 · ImageNet val (training with quantized activations) | memory reduction of stored activations: 6.1X -> 7.5X | Improved |
| start training in low precision (2,0) instead of full precision (ablation) | (F)-(3,2)-(2,0) schedule · ImageNet val (activation annealing, ResNet-50) | top-1 accuracy: 75.454 -> 50.360 | Hurt |
| anneal large activation ratio from 2% to 1% to 0% at 3 bits (ablation) | (3,2)-(3,2)-(3,2) · ImageNet val (activation annealing, ResNet-50) | top-1 accuracy: 75.720 vs similar to constant (3,2) | No clear effect |
| anneal further to 2-bit at the end (ablation) | (3,2)-(3,1)-(2,0) · ImageNet val (activation annealing, ResNet-50) | top-1 accuracy: 0.584% smaller accuracy | Hurt |
| replace checkpointing with RV-Quant (3,2) | ResNet-152 · ImageNet training activation memory | memory: 5.29GB -> 3.09GB | Improved |
| replace checkpointing with RV-Quant (3,2) | Inception-v3 · ImageNet training activation memory | memory: 3.87GB -> 1.79GB | Improved |
| replace checkpointing with RV-Quant | ResNet-50 training runtime · ResNet-50 batch 64 on M40 | runtime overhead: 32.4% -> 8.8% | Improved |
Closest papers by meaning
- LQ-Nets: Learned Quantization for Highly Accurate and Compact Deep Neural Networks (ECCV 2018 · similarity 0.89)
- Not All Bits have Equal Value: Heterogeneous Precisions via Trainable Noise (NeurIPS 2022 · similarity 0.89)
- LEARNED STEP SIZE QUANTIZATION (ICLR 2020 · similarity 0.89)
- Explicit Loss-Error-Aware Quantization for Low-Bit Deep Neural Networks (CVPR 2018 · similarity 0.89)
- Efficient Multi-bit Quantization Network Training via Weight Bias Correction and Bit-wise Coreset Sampling (NeurIPS 2025 · similarity 0.89)
- Towards Cheaper Inference in Deep Networks with Lower Bit-Width Accumulators (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
