Research map / Architectures and efficiency / Quantization
Two-Step Quantization for Low-Bit Neural Networks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace HWGQ activation quantization with sparse quantization at 62.5% sparsity (ablation) | AlexNet · ImageNet | Top-1 accuracy: 55.8 -> 59.0 | Helped |
| increase activation sparsity from 50% to 62.5% (ablation) | AlexNet sparse quantization · ImageNet | Top-1 accuracy: 58.5 -> 59.0 | Helped |
| increase activation sparsity further to 75% (ablation) | AlexNet sparse quantization · ImageNet | Top-1 accuracy: 59.0 -> 57.9 | Hurt |
| fine-tune after transformation-function learning with SQ-2 codes (ablation) | AlexNet TFL-SQ-2 · ImageNet | Top-1 accuracy: 55.1 -> 58.0 | Helped |
| initialize transformation-function learning with OTWA rather than random ternary variables (ablation) | AlexNet SQ-2 · ImageNet | Top-1 accuracy: about 28.5% top-1 accuracy with OTWA alone; about 1.3% advantage for asymmetric learning | Helped |
| use asymmetric rather than symmetric transformation-function learning to account for previous-layer quantization error (ablation) | AlexNet SQ-2 · ImageNet | Top-1 accuracy: about 1.3% | Helped |
| use Two-Step Quantization with 2-bit activations and ternary weights rather than HWGQ | AlexNet · ImageNet | Top-1 accuracy: 52.7 -> 58.0 | Helped |
| use Two-Step Quantization with 2-bit activations and ternary weights rather than a full-precision baseline | AlexNet · ImageNet | Top-1 accuracy: 58.5 -> 58.0 | Hurt |
| use Two-Step Quantization rather than the full-precision VGG-16 baseline | VGG-16 · ImageNet | Top-1 accuracy: 71.1 -> 69.1 | Hurt |
| use Two-Step Quantization rather than the full-precision VGG-Small baseline | VGG-Small · CIFAR-10 | error: 6.82 -> 6.51 | Helped |
Closest papers by meaning
- Quantization Networks (CVPR 2019 · similarity 0.89)
- ParetoQ: Improving Scaling Laws in Extremely Low-bit LLM Quantization (NeurIPS 2025 · similarity 0.88)
- SYQ: Learning Symmetric Quantization for Efficient Deep Neural Networks (CVPR 2018 · similarity 0.88)
- Don’t Waste Your Bits! Squeeze Activations and Gradients for Deep Neural Networks via TinyScript (ICML 2020 · similarity 0.88)
- QBB: Quantization with Binary Bases for LLMs (NeurIPS 2024 · similarity 0.88)
- PTMQ: Post-training Multi-Bit Quantization of Neural Networks (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.