Research map / Architectures and efficiency / Quantization
MQBench: Towards Reproducible and Deployable Model Quantization Benchmark
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Align training pipelines and hyperparameters across quantization algorithms | 4-bit QAT on ImageNet · Academic setting, ResNet-50 | Top-1 accuracy spread among algorithms: 1.1% | No clear effect |
| Use rule-based DoReFa instead of learned LSQ | ResNet-18, 4-bit academic QAT · ImageNet val | Top-1 accuracy: 70.7% vs 70.7% | No clear effect |
| Switch from graph 1 to graph 3 (ablation) | LSQ · ResNet-18, 4-bit QAT | Top-1 accuracy: 70.7 -> 70.3 | Hurt |
| Switch from graph 1 to graph 3 (ablation) | PACT · MobileNetV2, 4-bit QAT | Top-1 accuracy: 70.7 -> 67.8 | Hurt |
| Fold BN with strategy 4 (ablation) | LSQ · ResNet-18, 4-bit QAT | Top-1 accuracy: 70.7 -> 70.4 | Hurt |
| Fold BN with strategy 4 (ablation) | PACT · MobileNetV2, 4-bit QAT | Top-1 accuracy: 70.7 -> 60.8 | Hurt |
| Use BN folding strategy 1 without updating running statistics (ablation) | LSQ · 4-bit QAT on ResNet-18 and MobileNetV2 | Top-1 accuracy: 0.2%~0.3% | No clear effect |
| Synchronize BN statistics during distributed training (ablation) | PACT · ResNet-18, 4-bit QAT with BN folding | Top-1 accuracy: 2.3% | Helped |
| Add one-epoch linear learning-rate warm-up (ablation) | QAT with BN folding · ImageNet | Convergence | Helped |
| Increase weight decay on the LSQ scale parameter to 8e-5 (ablation) | LSQ on MobileNetV2 with TensorRT · ImageNet val | Accuracy: 2.3% | Helped |
| Use MSE rather than Quantile activation calibration, keeping MinMax weight calibration (ablation) | ResNet-18, 8-bit PTQ · ImageNet val | Accuracy: over 4.1% | Helped |
| Use MSE calibration for both activations and weights rather than MinMax (ablation) | EfficientNet-Lite0, 8-bit PTQ · ImageNet val | Accuracy: over 1.1% | Helped |
Closest papers by meaning
- GSQ: Highly-Accurate Low-Precision Scalar Quantization for LLMs via Gumbel-Softmax Sampling (NeurIPS 2026 · similarity 0.89)
- ParetoQ: Improving Scaling Laws in Extremely Low-bit LLM Quantization (NeurIPS 2025 · similarity 0.89)
- KV Cache is 1 Bit Per Channel: Efficient Large Language Model Inference with Coupled Quantization (NeurIPS 2024 · similarity 0.88)
- QuEST: Stable Training of LLMs with 1-Bit Weights and Activations (ICML 2025 · similarity 0.88)
- SqueezeLLM: Dense-and-Sparse Quantization (ICML 2024 · similarity 0.88)
- ResQ: Mixed-Precision Quantization of Large Language Models with Low-Rank Residuals (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.