Research map / Architectures and efficiency / Quantization
MagR: Weight Magnitude Reduction for Enhancing Post-Training Quantization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MagR, a non-linear preprocessing step that reduces per-channel weight magnitudes via an l_infinity-regularized least-squares problem solved by proximal gradient descent, improving RTN/OPTQ post-training quantization of LLaMA models with no inference overhead.
Problem
Post-training quantization of LLMs to low bit-widths (especially INT2/INT3) loses accuracy; existing linear-transformation preprocessing methods reduce weight range but add inference overhead.
Approach
Preprocess each linear layer's weights by solving a column-wise l_inf-regularized layer-reconstruction problem (solved by proximal gradient with l1-ball projection) so the weight range shrinks while layer outputs are preserved; the transform is non-linear and requires no inference-time changes.
Key results
- MagR+OPTQ with coordinate-descent refinement (MagR+OPTQ†) reaches 5.95 Wikitext2 perplexity on LLaMA2-70B at per-channel INT2, beating OmniQuant (7.81) and QuIP (6.33).
- MagR+OPTQ achieves the best W4A16 Wikitext2 perplexity on LLaMA2-7B (5.70 vs 5.83 for OPTQ and 6.15 for AWQ).
- MagR+RTN outperforms AWQ on per-channel W3A16 (e.g., LLaMA2-7B Wikitext2 8.66 vs 24.00).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add MagR preprocessing before OPTQ | OPTQ on LLaMA2-7B W3A16 · WikiText2 | perplexity: 8.37 -> 6.41 | Helped |
| add MagR preprocessing before OPTQ | OPTQ on LLaMA2-7B W3A16 · C4 | perplexity: 9.81 -> 8.23 | Helped |
| add MagR preprocessing before OPTQ | OPTQ on LLaMA2-7B W4A16 · WikiText2 | perplexity: 5.83 -> 5.70 | Helped |
| add MagR preprocessing to QuIP (ablation) | QuIP on LLaMA2-7B W2A16 · WikiText2 | perplexity: 27.13 -> 13.31 | Helped |
| set α from 0.001 to 0.0005 (ablation) | MagR+OPTQ on LLaMA2-7B W4A16 · Wikitext2 | perplexity: 5.70 -> 5.72 | Hurt |
| set α from 0.001 to 0.0001 (ablation) | MagR+OPTQ on LLaMA2-7B W4A16 · Wikitext2 | perplexity: 5.70 -> 5.78 | Hurt |
| set α from 0.001 to 0.00001 (ablation) | MagR+OPTQ on LLaMA2-7B W4A16 · Wikitext2 | perplexity: 5.70 -> 5.81 | Hurt |
| set α from 0.001 to 0.005 (ablation) | MagR+OPTQ on LLaMA2-7B W4A16 · Wikitext2 | perplexity: 5.70 -> 5.84 | Hurt |
| set β from 1 to 0.9 (ablation) | MagR+OPTQ on LLaMA2-7B W3A16 · Wikitext2 | perplexity: 6.43 -> 6.41 | Helped |
| set β from 1 to 0.8 (ablation) | MagR+OPTQ† on LLaMA2-7B W2A16 · Wikitext2 | perplexity: 16.99 -> 16.73 | Helped |
| set β from 1 to 0.9 (ablation) | MagR+OPTQ† on LLaMA2-7B W2A16 · Wikitext2 | perplexity: 16.99 -> 20.88 | Hurt |
Surprising findings
- MagR+OPTQ† at INT2 is competitive with or beats QuIP on perplexity but not on all zero-shot tasks, even though QuIP uses incoherence processing with inference overhead.
Limitations
- MagR+OPTQ† INT2 quantization requires long runtime (up to 31 hr for LLaMA2-70B) due to extra coordinate descent iterations.
- Per-channel INT2 results depend on tuned hyperparameters alpha and beta selected per bit-width.
- Zero-shot accuracy gap to QuIP remains at INT2 on 13B and 70B models.
Experimental setup
Datasets: WikiText2 C4
Benchmarks: WikiText2 C4 PIQA ARC-Easy ARC-Challenge Winogrande
Metrics: perplexity zero-shot accuracy runtime
Baselines: RTN OPTQ AWQ OmniQuant QuIP
Builds on: OPTQ AWQ QuIP OmniQuant
Closest papers by meaning
- Exploring Post-training Quantization in LLMs from Comprehensive Study to Low Rank Compensation (AAAI 2024 · similarity 0.87)
- QBB: Quantization with Binary Bases for LLMs (NeurIPS 2024 · similarity 0.87)
- Q-Palette: Fractional-Bit Quantizers Toward Optimal Bit Allocation for Efficient LLM Deployment (NeurIPS 2025 · similarity 0.87)
- A2Q+: Improving Accumulator-Aware Weight Quantization (ICML 2024 · similarity 0.87)
- AffineQuant: Affine Transformation Quantization for Large Language Models (ICLR 2024 · similarity 0.87)
- GSQ: Highly-Accurate Low-Precision Scalar Quantization for LLMs via Gumbel-Softmax Sampling (NeurIPS 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.