Research map / Architectures and efficiency / Quantization
LEARNED STEP SIZE QUANTIZATION
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LSQ instead of prior quantization methods | ResNet-18, ResNet-34, and ResNet-50 · ImageNet | top-1 accuracy: 67.6 vs 64.4; 71.6 vs 70.6; 73.7 vs 72.2 | Helped |
| Use 4-bit rather than 8-bit quantization | LSQ ResNet-18 and ResNet-34 · ImageNet | top-1 accuracy: 71.1 vs 71.1; 74.1 vs 74.1 | No clear effect |
| Reduce weight decay from 10^-4 to 0.25 × 10^-4 (ablation) | 2-bit ResNet-18 · ImageNet | top-1 accuracy: 66.9 -> 67.6 | Helped |
| Reduce weight decay from 10^-4 to 0.5 × 10^-4 (ablation) | 3-bit ResNet-18 · ImageNet | top-1 accuracy: 70.1 -> 70.2 | Helped |
| Scale step-size gradients by layer size and precision (ablation) | 2-bit ResNet-18 · ImageNet | top-1 accuracy: 64.2 -> 67.6 | Helped |
| Adjust only for layer size, not precision (ablation) | 2-bit ResNet-18 · ImageNet | top-1 accuracy: 67.6 -> 67.3 | Hurt |
| Use no step-size gradient scaling at the standard initial learning rate (ablation) | 2-bit ResNet-18 · ImageNet | convergence | Hurt |
| Multiply the step-size gradient scale by ten or divide it by ten (ablation) | 2-bit ResNet-18 · ImageNet | top-1 accuracy: 67.6 -> 67.4; 67.6 -> 67.3 | Hurt |
| Replace cosine decay with step-based learning-rate decay (ablation) | 2-bit LSQ ResNet-18 · ImageNet | top-1 accuracy: 67.6 -> 67.2 | Hurt |
| Add same-architecture full-precision teacher distillation (ablation) | 3-bit ResNet-50 · ImageNet | top-1 accuracy: 75.8 -> 76.9 | Helped |
| Distill a full-precision student from a full-precision teacher (ablation) | Full-precision student · ImageNet | accuracy | No clear effect |
| Learn task-loss-optimal step sizes rather than minimizing quantization error (ablation) | 2-bit ResNet-18 · ImageNet | quantization error: 50%; 63%; 64%; 47%; 28%; 46% | Mixed |
Closest papers by meaning
- Learning to Quantize Deep Networks by Optimizing Quantization Intervals With Task Loss (CVPR 2019 · similarity 0.90)
- Network Quantization With Element-Wise Gradient Scaling (CVPR 2021 · similarity 0.90)
- Analysis of Quantized Models (ICLR 2019 · similarity 0.90)
- Incremental Network Quantization: Towards Lossless CNNs with Low-precision Weights (ICLR 2017 · similarity 0.90)
- Not All Bits have Equal Value: Heterogeneous Precisions via Trainable Noise (NeurIPS 2022 · similarity 0.90)
- Efficient Multi-bit Quantization Network Training via Weight Bias Correction and Bit-wise Coreset Sampling (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.