Research map / Architectures and efficiency / Quantization
LiftQuant: Continuous Bit-Width LLM via Dimensional Lifting and Projection
Liulu He, Xuan Ang Liu, Juntao Liu, Taolue Feng, Ting Lu, Chunsheng Gan, ZHIYV PENG, Yuan Du, Huanrui Yang, Yijiang Liu, Li Du
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add scaling s1 (ablation) | LiftQuant on Llama-2-7B · WikiText-2 | perplexity: 8.76 -> 8.28 | Improved |
| add scaling s2 (ablation) | LiftQuant on Llama-2-7B · WikiText-2 | perplexity: 8.28 -> 7.77 | Improved |
| add lifted projection matrix M instead of 2-bit uniform quantizer (ablation) | LiftQuant on Llama-2-7B · WikiText-2 | perplexity: 7.77 -> 6.79 | Improved |
| add end-to-end fine-tuning (ablation) | LiftQuant on Llama-2-7B · WikiText-2 | perplexity: 6.79 -> 6.53 | Improved |
| remove orthogonal mixing matrices P1 and P2 (ablation) | LiftQuant whitening · Llama-2-7B | training stability | Hurt |
| increase bit-width from 2.0 to 2.14 (LQ-32/16 to LQ-30/14) | LiftQuant · Llama-2-7B WikiText-2 | perplexity: 6.53 -> 6.30 | Improved |
| increase bit-width to 2.4 (LQ-24/10) from 2-bit LQ-32/16 | LiftQuant · Llama-3-70B WikiText-2 | perplexity: 4.69 -> 4.10 | Improved |
| increase bit-width to 2.4 (LQ-24/10) from 2-bit LQ-32/16 | LiftQuant · Llama-2-70B WikiText-2 | perplexity: 4.08 -> 3.78 | Improved |
| increase lifted dimension from LQ-16/8 to LQ-32/16 at 2 bits | LiftQuant · Standard Gaussian / Llama-2-7B | perplexity: 6.60 -> 6.53 | Improved |
| increase search dimension gap (LQ-32/16 vs LQ-16/8) | LiftQuant · nearest-neighbor search time | search time: <<0.1S -> 4S | Hurt |
| use LiftQuant 2-bit instead of QTIP | LiftQuant LQ-32/16 · Llama-2-7B WikiText-2 | perplexity: 6.28 -> 6.52 | Hurt |
| use LiftQuant 2-bit instead of QTIP | LiftQuant LQ-32/16 · Llama-3-70B WikiText-2 | perplexity: 4.97 -> 4.69 | Improved |
Closest papers by meaning
- KVQuant: Towards 10 Million Context Length LLM Inference with KV Cache Quantization (NeurIPS 2024 · similarity 0.86)
- Q-Palette: Fractional-Bit Quantizers Toward Optimal Bit Allocation for Efficient LLM Deployment (NeurIPS 2025 · similarity 0.86)
- QuEST: Stable Training of LLMs with 1-Bit Weights and Activations (ICML 2025 · similarity 0.86)
- QuIP$\#$: Even Better LLM Quantization with Hadamard Incoherence and Lattice Codebooks (ICML 2024 · similarity 0.86)
- LeanQuant: Accurate and Scalable Large Language Model Quantization with Loss-error-aware Grid (ICLR 2025 · similarity 0.86)
- NSNQuant: A Double Normalization Approach for Calibration-Free Low-Bit Vector Quantization of KV Cache (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
