Research map / Architectures and efficiency / Quantization
Network Quantization With Element-Wise Gradient Scaling
Check your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace STE with EWGS gradient scaling (ablation) | ResNet-18 quantization · ImageNet, W/A 1/1 | top-1 accuracy: 54.6 -> 55.3 | Helped |
| replace STE with EWGS gradient scaling (ablation) | ResNet-18 quantization · ImageNet, W/A 1/32 | top-1 accuracy: 66.3 -> 67.3 | Helped |
| replace STE with EWGS gradient scaling (ablation) | MobileNet-V2 quantization · ImageNet, W/A 4/4 | top-1 accuracy: 69.2 -> 70.3 | Helped |
| replace STE with EWGS gradient scaling (ablation) | ResNet-20 quantization · CIFAR-10, W/A 1/1 | top-1 accuracy: 84.7 -> 85.6 | Helped |
| use EWGS in place of STE (ablation) | DoReFa quantization · CIFAR-10, ResNet-20, W/A 1/1 | top-1 accuracy: 84.9 -> 85.9 | Helped |
| use EWGS in place of STE (ablation) | DoReFa quantization · CIFAR-10, ResNet-20, W/A 1/32 | top-1 accuracy: 89.7 -> 90.3 | Helped |
| use a Hessian-derived scaling factor with G=3σ instead of fixed scaling (ablation) | ResNet-20 binary quantization · CIFAR-10 | top-1 accuracy: 85.3 -> 85.6 | Helped |
| set the fixed EWGS scaling factor to 1e-1 (ablation) | ResNet-20 binary quantization · CIFAR-10 | top-1 accuracy: 85.6 -> 60.9 | Hurt |
| use mean absolute gradient as the scaling representative G instead of 3σ (ablation) | ResNet-20 binary quantization · CIFAR-10 | top-1 accuracy: 85.6 -> 83.1 | Hurt |
| replace EWGS with STE (ablation) | ResNet-20 binary quantization · CIFAR-10 | top-1 accuracy: 85.6 -> 84.7 | Hurt |
| replace EWGS with STE (ablation) | PROFIT quantization · ImageNet, MobileNet-V2, W/A 4/4 | top-1 accuracy: 69.2 -> 70.0 | Helped |
| use EWGS rather than STE for gradient propagation (ablation) | quantized networks across tested architectures and methods · ImageNet and CIFAR-10 | training stability and accuracy | Helped |
Closest papers by meaning
- Analysis of Quantized Models (ICLR 2019 · similarity 0.91)
- LEARNED STEP SIZE QUANTIZATION (ICLR 2020 · similarity 0.90)
- A Statistical Framework for Low-bitwidth Training of Deep Neural Networks (NeurIPS 2020 · similarity 0.90)
- Explicit Loss-Error-Aware Quantization for Low-Bit Deep Neural Networks (CVPR 2018 · similarity 0.89)
- One-Step Forward and Backtrack: Overcoming Zig-Zagging in Loss-Aware Quantization Training (AAAI 2024 · similarity 0.89)
- QBB: Quantization with Binary Bases for LLMs (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.