Research map / Architectures and efficiency / Quantization

ASER: Activation Smoothing and Error Reconstruction for Large Language Model Quantization

AAAI 2025 · Architectures and efficiency › Quantization · approach: Post-training quantization of LLMs · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ASER, a post-training quantization method that compensates quantization error with low-rank LoRA-style matrices from whitening SVD and smooths activation outliers, evaluated on LLaMA3-8B, Qwen1.5-7B and Qwen-72B under W4A8 and W4A6.

Problem

Low-bit post-training quantization of LLMs produces large layer-wise output error (||WX - W_q X||), causing intolerable accuracy degradation, especially when activations are also quantized.

Approach

Treat quantization as minimizing output discrepancy; whiten the calibration activations via Cholesky factorization so that SVD of the weight error times the whitening matrix gives singular values equal to output-error contributions, then reconstruct the error with a rank-r LoRA-style pair. Outlier channels are migrated from activations to weights and kept unquantized so they are compensated by the

Key results

• On LLaMA3-8B W4A6, ASER (w/ A.S.) reaches avg accuracy 48.55% vs 43.68% for L2QER and WikiText2 perplexity 8.41 vs 9.44.
• On Qwen1.5-7B W4A8, ASER (w/ A.S.) reaches avg accuracy 65.30% vs 59.50% for L2QER, with WikiText2 perplexity 8.72 vs fp16 7.95.
• Overhead analysis on Qwen-7B shows the rank configuration with alpha=0.1 adds about 2.93% FLOPs, and alpha=0.015 adds 0.26%.

Measured effects reported in the paper

Surprising findings

• On Qwen1.5-7B, ASER without activation smoothing gets a higher avg accuracy than the variant with it at W4A6 on some tasks, and larger rank is not always better for downstream tasks.
• SmoothQuant and SmoothQuant+ collapse under W4A6 per-channel quantization on Qwen1.5-7B (SmoothQuant+ WikiText2 PPL 131.45).

Limitations

• Adaptive selection of rank threshold alpha is left for future work; larger rank does not uniformly improve downstream tasks.
• Activation smoothing relies on a hand-designed heuristic outlier threshold f, fixed at 32 channels.
• Evaluation is limited to RTN-based and a few PTQ baselines on three models; no code or data release is stated.

Experimental setup

Datasets: calibration data (unspecified) WikiText2 PTB C4

Benchmarks: WikiText2 PTB C4 ARC-e ARC-c MMLU HellaSwag PIQA GSM8K HumanEval

Metrics: perplexity accuracy average accuracy FLOPs overhead

Baselines: LLM.int4() SmoothQuant SmoothQuant+ LoRC L2QER

Builds on: SmoothQuant LoRC L2QER LoRA GPTQ

Closest papers by meaning

• LQER: Low-Rank Quantization Error Reconstruction for LLMs (ICML 2024 · similarity 0.93)
• Norm Tweaking: High-Performance Low-Bit Quantization of Large Language Models (AAAI 2024 · similarity 0.92)
• Quantization Error Propagation: Revisiting Layer-Wise Post-Training Quantization (NeurIPS 2025 · similarity 0.91)
• Exploring Post-training Quantization in LLMs from Comprehensive Study to Low Rank Compensation (AAAI 2024 · similarity 0.91)
• What Makes Quantization for Large Language Model Hard? An Empirical Study from the Lens of Perturbation (AAAI 2024 · similarity 0.91)
• Rethinking Channel Dimensions to Isolate Outliers for Low-bit Weight Quantization of Large Language Models (ICLR 2024 · similarity 0.90)