Research map / Architectures and efficiency / Quantization
FLRQ: Faster LLM Quantization with Flexible Low-Rank Matrix Sketching
Hongyaoxing Gu, Lijuan Hu, Shuzi Niu, Fangfang Liu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add BLC iterative low-rank/clipping refinement (ablation) | FLRQ · WikiText2 W2A16 | perplexity: 2.1e6 -> 9.14 (LLaMA2-7b) | Improved |
| add BLC (ablation) | FLRQ · WikiText2 W4A16 | perplexity: 5.55 -> 5.55 (LLaMA2-7b) | No clear effect |
| use flexible rank (R1-FLR) instead of fixed rank 64 (ablation) | FLRQ · WikiText2 W4A16 LLaMA2-7b | perplexity: 5.73 -> 5.74 | No clear effect |
| use FLRQ vs OmniQuant at 2-bit | LLaMA2-7b · WikiText2 W2A16 | perplexity: 11.06 -> 9.14 | Improved |
| use FLRQ vs OmniQuant at 4-bit | LLaMA2-7b · WikiText2 W4A16 | perplexity: 5.58 -> 5.55 | No clear effect |
| FLRQ with flexible avg rank ~39 vs LQER rank 256 | LLaMA2-7b · WikiText2 W2A16 | perplexity: 10.33 -> 9.14 | Improved |
| FLRQ vs LQER on C4 at 2-bit | LLaMA2-7b · C4 W2A16 | perplexity: 12.12 -> 12.10 | Mixed |
| use R1-Sketch instead of exact SVD (ablation) | FLRQ · WikiText2 W3A16 | perplexity: 15.53 -> 15.53 (OPT-1.3B); time 8.7m -> 6.1m | No clear effect |
| increase sketch iterations beyond 2 (ablation) | FLRQ · WikiText2 W3A16 | perplexity: 15.53 -> 15.53 (it 2 -> 8) | No clear effect |
| add power iterations (it=0 to 2) (ablation) | R1-Sketch · WikiText2 W3A16 OPT-1.3B | perplexity: 16.84 -> 15.53 | Improved |
| use FLRQ vs OmniQuant quantization time at 2-bit | LLaMA2-7b · quantization time | hours: 3.1h -> 2.0h | Improved |
| use FLRQ vs OmniQuant zero-shot at 2-bit | LLaMA2-7b · 6-task zero-shot avg | accuracy: 50.1% -> 56.4% | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 2 of 2 on most shared results.
- OmniQuant — OmniQuant: Omnidirectionally Calibrated Quantization for Large Language Models (ICLR 2024) · better on 8 of 8 shared results
- AffineQuant — AffineQuant: Affine Transformation Quantization for Large Language Models (ICLR 2024) · better on 3 of 3 shared results
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- ResQ: Mixed-Precision Quantization of Large Language Models with Low-Rank Residuals (ICML 2025 · similarity 0.92)
- LQER: Low-Rank Quantization Error Reconstruction for LLMs (ICML 2024 · similarity 0.91)
- Exploring Post-training Quantization in LLMs from Comprehensive Study to Low Rank Compensation (AAAI 2024 · similarity 0.91)
- QLLM: Accurate and Efficient Low-Bitwidth Quantization for Large Language Models (ICLR 2024 · similarity 0.91)
- RILQ: Rank-Insensitive LoRA-Based Quantization Error Compensation for Boosting 2-Bit Large Language Model Accuracy (AAAI 2025 · similarity 0.91)
- OWQ: Outlier-Aware Weight Quantization for Efficient Fine-Tuning and Inference of Large Language Models (AAAI 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
