AI Paper Atlas

Research map / Architectures and efficiency / Quantization

Intriguing Properties of Quantization at Scale

Arash Ahmadian, Saurabh Dash, Hongyu Chen, Bharat Venkitesh, Zhen Stephen Gou, Phil Blunsom, Ahmet Üstün, Sara Hooker

NeurIPS 2023 · Architectures and efficiency › Quantization · approach: Post-training weight quantization

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Increase weight decay from 0.001 to 0.1 (ablation)6B model trained without gradient clipping · 8-task zero-shot averagequantization degradation: 1.36% vs 0.09% degradationImproved
Add gradient clipping at 1.0 (ablation)6B model with weight decay 0.001 · 8-task zero-shot averagequantization robustness: -1.36 -> 0.41Improved
Train with bf16 rather than fp16 (ablation)6B model with weight decay 0.1 · 8-task zero-shot averagequantization robustness: -0.76 -> 0.32Improved
Train with fp16 rather than bf16 at weight decay 0.01 (ablation)6B model · 8-task zero-shot averagequantization robustness: 0.77 -> -1.73Hurt
Increase dropout from 0.0 to 0.4 (ablation)6B model · 8-task zero-shot averagequantization robustness: 0.32 -> -0.27Hurt
Use dropout 0.8 instead of 0.0 (ablation)6B model · 8-task zero-shot averagepre-quantization performance: 66.11 -> 50.93Hurt
Apply INT8 weight-and-activation PTQ52B model trained with weight decay 0.1, gradient clipping 1, no dropout, and bf16 · 8-task zero-shot averagemean zero-shot performance: 77.31 -> 77.39Improved
Apply INT8 weight-and-activation PTQ13B model trained with the recommended recipe · 8-task zero-shot averagemean zero-shot performance: 72.83 -> 72.59Hurt
Apply INT8 weight-and-activation PTQ6B model trained with the recommended recipe · 8-task zero-shot averagemean zero-shot performance: 72.67 -> 72.18Hurt
Apply INT8 weight-and-activation PTQ410M model trained with the recommended recipe · 8-task zero-shot averagemean zero-shot performance: 59.36 -> 58.85Hurt
Apply INT4 column-wise weight-only quantization52B model trained with the recommended recipe · 8-task zero-shot averagemean zero-shot performance: 77.31 -> 73.72Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy