Research map / Architectures and efficiency / Quantization
ZeroQuant: Efficient and Affordable Post-Training Quantization for Large-Scale Transformers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add group-wise weight quantization (ablation) | W4/8A8 PTQ on BERT-large · GLUE dev | average score: 35.85 -> 66.52 | Helped |
| add token-wise activation quantization on top of group-wise weights (ablation) | ZeroQuant on BERT-large W4/8A8 · GLUE dev | average score: 66.52 -> 81.06 | Helped |
| add layer-by-layer distillation (LKD) (ablation) | ZeroQuant on BERT-large W4/8A8 · GLUE dev | average score: 81.06 -> 81.62 | Helped |
| use ZeroQuant fine-grained quantization instead of PTQ at W8A8 (ablation) | BERT-base · GLUE dev | average score: 77.41 -> 83.75 | Helped |
| add LKD to ZeroQuant at W4/8A16 | BERT-base · GLUE dev | average score: 81.65 -> 82.35 | Helped |
| add LKD to ZeroQuant at W4/8A16 | GPT-3 350M style · Wikitext-2 | perplexity: 88.6 -> 30.6 | Helped |
| add LKD to ZeroQuant at W4/8A16 | GPT-3 350M style · 19-task average | accuracy: 33.5 -> 37.0 | Helped |
| distill with random token data instead of no LKD (ablation) | GPT-3 350M style W4/8A8 · Wikitext-2 | perplexity: 92.1 -> 40.6 | Helped |
| distill with Wikipedia instead of original data (ablation) | GPT-3 350M style W4/8A8 · 19-task average | accuracy: 36.6 -> 36.2 | No clear effect |
| use Wikipedia rather than random data for LKD (ablation) | GPT-3 350M style W4/8A8 · Wikitext-2 | perplexity: 40.6 -> 30.4 | Helped |
| quantize attention-input activations to INT8 in GPT-NeoX-20B | GPT-NeoX-20B · 19 zero-shot tasks | accuracy | Hurt |
| add LKD to ZeroQuant on BERT-large at W4/8A16 for RTE | BERT-large · RTE | accuracy: 48.01 -> 51.62 | No clear effect |
Closest papers by meaning
- GuidedQuant: Large Language Model Quantization via Exploiting End Loss Guidance (ICML 2025 · similarity 0.90)
- FrameQuant: Flexible Low-Bit Quantization for Transformers (ICML 2024 · similarity 0.90)
- OSTQuant: Refining Large Language Model Quantization with Orthogonal and Scaling Transformations for Better Distribution Fitting (ICLR 2025 · similarity 0.89)
- AffineQuant: Affine Transformation Quantization for Large Language Models (ICLR 2024 · similarity 0.89)
- LeanQuant: Accurate and Scalable Large Language Model Quantization with Loss-error-aware Grid (ICLR 2025 · similarity 0.89)
- Norm Tweaking: High-Performance Low-Bit Quantization of Large Language Models (AAAI 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.