Research map / Architectures and efficiency / Quantization
OSAQ: Outlier Self-Absorption for Accurate Low-bit LLM Quantization
Zhikai Li, Zhen Dong, Xuewen Liu, Jing Zhang, Qingyi Gu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add OSAQ additive transformation before GPTQ at 2-bit | GPTQ · WikiText2 W2A16 g128, LLaMA2-7B | perplexity: 36.8 -> 21.2 | Improved |
| add coordinate descent iterations on top of OSAQ+GPTQ at 2-bit | OSAQ+GPTQ · WikiText2 W2A16 g128, LLaMA2-7B | perplexity: 21.2 -> 10.6 | Improved |
| add OSAQ before GPTQ at 3-bit per-channel | GPTQ · WikiText2 W3A16, LLaMA2-13B | perplexity: 6.44 -> 5.72 | Improved |
| add OSAQ before GPTQ at 4-bit | GPTQ · WikiText2 W4A16, LLaMA2-7B | perplexity: 5.83 -> 5.73 | Improved |
| add OSAQ before AWQ | AWQ · WikiText2 W3A16 g128, LLaMA2-7B | perplexity: 6.24 -> 6.08 | Improved |
| add OSAQ before QuIP | QuIP · WikiText2 W3A16, LLaMA2-7B | perplexity: 6.50 -> 6.37 | Improved |
| add OSAQ before GPTQ at 3-bit | GPTQ · Commonsense QA average, LLaMA3-8B, W3A16 g128 | zero-shot accuracy: 63.6 -> 65.2 | Improved |
| add OSAQ before GPTQ at 4-bit | GPTQ · MMLU, LLaMA2-7B and LLaMA3-8B, W4A16 | average accuracy: 38.7 -> 39.6; 57.6 -> 58.2 | Improved |
| add OSAQ before GPTQ at 3-bit on MMLU | GPTQ · MMLU, LLaMA3-8B, W3A16 | average accuracy: 30.0 -> 30.9 | No clear effect |
| add OSAQ before GPTQ on very large instruction-tuned models | GPTQ · Mistral-Large-123B-Instruct W4A16, MMLU average | accuracy: 82.0 -> 82.3 | Improved |
| apply additive transformation to the FP16 model without quantization (ablation) | LLaMA2-7B FP16 · WikiText2 | perplexity: 5.47 -> 5.52 | No clear effect |
| use Softmax-inf approximation instead of plain l2 norm objective (ablation) | OSAQ+GPTQ · WikiText2 W3A16, LLaMA2-7B | perplexity: 7.82 -> 6.75 | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 2 of 2 on most shared results.
- OmniQuant — OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models (ICLR 2024) · better on 3 of 3 shared results
- QuIP — QuIP: 2-Bit Quantization of Large Language Models With Guarantees (NeurIPS 2023) · better on 1 of 1 shared result
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Rethinking Channel Dimensions to Isolate Outliers for Low-bit Weight Quantization of Large Language Models (ICLR 2024 · similarity 0.93)
- OWQ: Outlier-Aware Weight Quantization for Efficient Fine-Tuning and Inference of Large Language Models (AAAI 2024 · similarity 0.92)
- Treasures in Discarded Weights for LLM Quantization (AAAI 2025 · similarity 0.92)
- DuQuant: Distributing Outliers via Dual Transformation Makes Stronger Quantized LLMs (NeurIPS 2024 · similarity 0.92)
- BiLLM: Pushing the Limit of Post-Training Quantization for LLMs (ICML 2024 · similarity 0.91)
- Exploring Post-training Quantization in LLMs from Comprehensive Study to Low Rank Compensation (AAAI 2024 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
