Research map / Architectures and efficiency / Quantization
Intriguing Properties of Quantization at Scale
Arash Ahmadian, Saurabh Dash, Hongyu Chen, Bharat Venkitesh, Zhen Stephen Gou, Phil Blunsom, Ahmet Üstün, Sara Hooker
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase weight decay from 0.001 to 0.1 (ablation) | 6B model trained without gradient clipping · 8-task zero-shot average | quantization degradation: 1.36% vs 0.09% degradation | Improved |
| Add gradient clipping at 1.0 (ablation) | 6B model with weight decay 0.001 · 8-task zero-shot average | quantization robustness: -1.36 -> 0.41 | Improved |
| Train with bf16 rather than fp16 (ablation) | 6B model with weight decay 0.1 · 8-task zero-shot average | quantization robustness: -0.76 -> 0.32 | Improved |
| Train with fp16 rather than bf16 at weight decay 0.01 (ablation) | 6B model · 8-task zero-shot average | quantization robustness: 0.77 -> -1.73 | Hurt |
| Increase dropout from 0.0 to 0.4 (ablation) | 6B model · 8-task zero-shot average | quantization robustness: 0.32 -> -0.27 | Hurt |
| Use dropout 0.8 instead of 0.0 (ablation) | 6B model · 8-task zero-shot average | pre-quantization performance: 66.11 -> 50.93 | Hurt |
| Apply INT8 weight-and-activation PTQ | 52B model trained with weight decay 0.1, gradient clipping 1, no dropout, and bf16 · 8-task zero-shot average | mean zero-shot performance: 77.31 -> 77.39 | Improved |
| Apply INT8 weight-and-activation PTQ | 13B model trained with the recommended recipe · 8-task zero-shot average | mean zero-shot performance: 72.83 -> 72.59 | Hurt |
| Apply INT8 weight-and-activation PTQ | 6B model trained with the recommended recipe · 8-task zero-shot average | mean zero-shot performance: 72.67 -> 72.18 | Hurt |
| Apply INT8 weight-and-activation PTQ | 410M model trained with the recommended recipe · 8-task zero-shot average | mean zero-shot performance: 59.36 -> 58.85 | Hurt |
| Apply INT4 column-wise weight-only quantization | 52B model trained with the recommended recipe · 8-task zero-shot average | mean zero-shot performance: 77.31 -> 73.72 | Hurt |
Closest papers by meaning
- Scaling Laws for Precision (ICLR 2025 · similarity 0.88)
- What Makes Quantization for Large Language Model Hard? An Empirical Study from the Lens of Perturbation (AAAI 2024 · similarity 0.87)
- Unified Scaling Laws for Compressed Representations (NeurIPS 2025 · similarity 0.87)
- Predicting Emergent Abilities with Infinite Resolution Evaluation (ICLR 2024 · similarity 0.87)
- Rethinking Channel Dimensions to Isolate Outliers for Low-bit Weight Quantization of Large Language Models (ICLR 2024 · similarity 0.87)
- QuantDemoire: Quantization with Outlier Aware for Image Demoiréing (NeurIPS 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
