Research map / Architectures and efficiency / Quantization
Q-BERT: Hessian Based Ultra Low Precision Quantization of BERT
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Hessian-based 2/3-bit mixed precision rather than uniform 2-bit weights | Q-BERT on SST-2 · SST-2 dev | accuracy: 84.63 -> 92.08 | Helped |
| Use Hessian-based 2/3-bit mixed precision rather than uniform 2-bit weights | Q-BERT on MNLI-m · MNLI-m dev | accuracy: 76.56 -> 81.75 | Helped |
| Use Hessian-based 2/3-bit mixed precision rather than uniform 2-bit weights | Q-BERT on CoNLL-03 · CoNLL-03 dev | F1: 91.06 -> 94.37 | Helped |
| Use Hessian-based 2/3-bit mixed precision rather than uniform 2-bit weights | Q-BERT on SQuAD · SQuAD dev | F1: 79.60 -> 86.95 | Helped |
| Add group-wise quantization, using 128 groups instead of layer-wise quantization (ablation) | Q-BERT on SST-2 · SST-2 dev | accuracy: 85.67 -> 92.66 | Helped |
| Add group-wise quantization, using 128 groups instead of layer-wise quantization (ablation) | Q-BERT on MNLI-m · MNLI-m dev | accuracy: 76.69 -> 83.89 | Helped |
| Add group-wise quantization, using 128 groups instead of layer-wise quantization (ablation) | Q-BERT on CoNLL-03 · CoNLL-03 dev | F1: 89.86 -> 94.90 | Helped |
| Increase group count from 128 to 768 (ablation) | Q-BERT on SST-2 · SST-2 dev | accuracy: 92.66 -> 92.78 | No clear effect |
| Increase group count from 128 to 768 (ablation) | Q-BERT on MNLI-m · MNLI-m dev | accuracy: 83.89 -> 84.00 | No clear effect |
| Increase group count from 128 to 768 (ablation) | Q-BERT on CoNLL-03 · CoNLL-03 dev | F1: 94.90 -> 94.99 | No clear effect |
| Quantize position rather than word embeddings to 4 bits (ablation) | Q-BERT on SST-2 · SST-2 dev | accuracy: 91.74 -> 89.11 | Hurt |
| Quantize the fully connected module more aggressively than the self-attention module (ablation) | Q-BERT mixed-precision encoder · SST-2 | accuracy: 89.56 -> 85.89 | Hurt |
Closest papers by meaning
- QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Models (ICLR 2024 · similarity 0.87)
- SliM-LLM: Salience-Driven Mixed-Precision Quantization for Large Language Models (ICML 2025 · similarity 0.86)
- GuidedQuant: Large Language Model Quantization via Exploiting End Loss Guidance (ICML 2025 · similarity 0.86)
- Norm Tweaking: High-Performance Low-Bit Quantization of Large Language Models (AAAI 2024 · similarity 0.86)
- ASER: Activation Smoothing and Error Reconstruction for Large Language Model Quantization (AAAI 2025 · similarity 0.85)
- Delta-CoMe: Training-Free Delta-Compression with Mixed-Precision for Large Language Models (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.