Research map / Architectures and efficiency / Quantization
PsumQuant: In-line Post-training Partial Sum Quantizer for Energy Efficient NPU Inference
Sangwoo Hwang, Yeeun Hong, Jaeha Kung
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use PsumQuant instead of static psum quantization | W8A8 SmoothQuant Llama 3.1 (8B) · WikiText-2, 128x128 array | Perplexity: 6.46 -> 6.14 | Improved |
| use PsumQuant instead of static psum quantization | W8A8 Phi-3 (3.8B) · WikiText-2, 128x128 array | Perplexity: 6.54 -> 6.27 | Improved |
| use PsumQuant instead of static psum quantization | W8A8 Qwen-2.5 (1.5B) · WikiText-2, 128x128 array | Perplexity: 9.17 -> 8.87 | Improved |
| use PsumQuant instead of static psum quantization | W8A8 Llama 3.1 (8B) · Zero-shot average of 5 tasks | Average accuracy (%): 66.1 -> 66.4 | Improved |
| use PsumQuant instead of static psum quantization | ResNet-18, B_PQ=6 · ImageNet Top-1 | Top-1 accuracy (%): 67.66 -> 70.04 | Improved |
| use PsumQuant instead of static psum quantization | ResNet-34, B_PQ=6 · ImageNet Top-1 | Top-1 accuracy (%): 73.97 -> 74.57 | Improved |
| use PsumQuant instead of static psum quantization | Swin-T, B_PQ=6 · ImageNet Top-1 | Top-1 accuracy (%): 78.78 -> 81.02 | Improved |
| add crest-factor (kappa_nz) gradient modulation to scale predictor calibration (ablation) | PsumQuant, Llama 3.1 · WikiText-2 | Perplexity / max absolute error | Improved |
| use PsumQuant instead of static scaling for scale-psum correlation | ResNet-18 · Scale vs max absolute psum | Pearson correlation r: 0.32 -> 0.92 | Improved |
| reduce psum bit-width from 24-bit to 8-bit with PsumQuant | 128x128 weight-stationary array · Total average energy | energy reduction: 45% reduction | Improved |
| use PTQ-based PsumQuant calibration instead of QAT | PsumQuant · Calibration latency | latency: 3.5x lower | Improved |
| reduce NPU array size from 256x256 to 64x64 | PsumQuant calibration, Llama 3.1 8B · Calibration time | Time (min): 18.92 -> 55.97 | Hurt |
Closest papers by meaning
- Towards Cheaper Inference in Deep Networks with Lower Bit-Width Accumulators (ICLR 2024 · similarity 0.87)
- A2Q+: Improving Accumulator-Aware Weight Quantization (ICML 2024 · similarity 0.86)
- KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization (NeurIPS 2024 · similarity 0.86)
- ParetoQ: Improving Scaling Laws in Extremely Low-bit LLM Quantization (NeurIPS 2025 · similarity 0.86)
- SpinQuant: LLM Quantization with Learned Rotations (ICLR 2025 · similarity 0.85)
- CodeGEMM: A Codebook-Centric Approach to Efficient GEMM in Quantized LLMs (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
