Research map / Architectures and efficiency / Quantization
PV-Tuning: Beyond Straight-Through Estimation for Extreme LLM Compression
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes PV-Tuning, a representation-agnostic fine-tuning framework that alternates continuous parameter updates and discrete code updates via a linearized subspace V step, enabling extreme (1-2 bit) LLM compression that beats STE-based fine-tuning.
Problem
Fine-tuning heavily quantized LLM weights with discrete parameters at 1-2 bits per parameter: straight-through estimators are heuristic and unstable, and fine-tuning only continuous parameters is limited.
Approach
Alternate a P step (backprop-based update of continuous codebooks/scales) with a V step that linearizes the loss and updates only a greedily chosen subspace of discrete codes (top-tau by Adam update magnitude), so large effective updates can be made without straight-through estimation; the framework also comes with a monotone-loss convergence result.
Key results
- On Llama 2 7B with VQ, PV-Tuning+STE reaches WikiText-2 perplexity 7.32 vs 7.76 for STE and 8.79 for GPTQ+STE; similar gains on AQLM (5.90 vs 6.41).
- At about 2 bits, PV-Tuning on Llama 2 70B reaches WikiText-2 perplexity 3.78 at 2.07 avg bits, versus 3.91 for QuIP# at 2.01 bits.
- PV-Tuning claims to push the Pareto-optimal frontier for Llama 2 to 2 bits per parameter, with a 2-bit 13B model outperforming any 7B quantization.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use subspace linearized PV instead of straight-through estimation | GPTQ on Llama 2 7B · WikiText-2 | perplexity: 8.79 -> 8.49 | Helped |
| Use subspace linearized PV instead of straight-through estimation | VQ on Llama 2 7B · WikiText-2 | perplexity: 7.76 -> 7.38 | Helped |
| Use subspace linearized PV instead of straight-through estimation | AQLM on Llama 2 7B · WikiText-2 | perplexity: 6.41 -> 6.13 | Helped |
| Combine subspace linearized PV with STE (ablation) | GPTQ on Llama 2 7B · WikiText-2 | perplexity: 8.49 -> 8.43 | Helped |
| Combine subspace linearized PV with STE (ablation) | VQ on Llama 2 7B · WikiText-2 | perplexity: 7.38 -> 7.32 | Helped |
| Combine subspace linearized PV with STE (ablation) | AQLM on Llama 2 7B · WikiText-2 | perplexity: 6.13 -> 5.90 | Helped |
| Use subspace linearized PV instead of continuous-parameter-only fine-tuning | GPTQ on Llama 2 7B · WikiText-2 | perplexity: 16.77 -> 8.49 | Helped |
| Use subspace linearized PV instead of continuous-parameter-only fine-tuning | VQ on Llama 2 7B · WikiText-2 | perplexity: 8.17 -> 7.38 | Helped |
| Use subspace linearized PV instead of continuous-parameter-only fine-tuning | AQLM on Llama 2 7B · WikiText-2 | perplexity: 6.69 -> 6.13 | Helped |
| Use PV-Tuning instead of QuIP# at roughly 2 bits | Llama 2 7B · Average zero-shot tasks | accuracy: 57.51 -> 61.35 | Helped |
| Use PV-Tuning instead of DB-LLM at roughly 2 bits | Llama 3 8B · Average zero-shot tasks | accuracy: 51.8 -> 64.36 | Helped |
Surprising findings
- Straight-through estimation initially improves the solution but then stops improving and oscillates, and stochastic rounding converges to significantly worse solutions at low bit-widths.
- Outliers yield a much smaller perplexity improvement when the quantized model is also fine-tuned.
Limitations
- Hyperparameters (constant learning rate, tau) were lightly tuned, so results may improve with better settings.
- Requires more compute and memory than fine-tuning only continuous parameters, and 70B runs need 8xA100 or offloading.
- Subspace selection is a greedy heuristic without theoretical justification; convergence guarantee only shows monotone loss, not convergence to a minimizer.
Experimental setup
Datasets: RedPajama WikiText-2 C4
Benchmarks: WikiText-2 C4 WinoGrande PIQA HellaSwag ARC-easy ARC-challenge
Metrics: perplexity zero-shot accuracy L2 error average bits per parameter
Baselines: GPTQ AQLM QuIP# QuIP BiLLM PB-LLM DB-LLM OneBit Straight-through estimation Stochastic rounding
Builds on: AQLM QuIP# GPTQ straight-through estimator Adam
Closest papers by meaning
- Delta-CoMe: Training-Free Delta-Compression with Mixed-Precision for Large Language Models (NeurIPS 2024 · similarity 0.91)
- LittleBit: Ultra Low-Bit Quantization via Latent Factorization (NeurIPS 2025 · similarity 0.91)
- SqueezeLLM: Dense-and-Sparse Quantization (ICML 2024 · similarity 0.91)
- SpQR: A Sparse-Quantized Representation for Near-Lossless LLM Weight Compression (ICLR 2024 · similarity 0.91)
- BitStack: Any-Size Compression of Large Language Models in Variable Memory Environments (ICLR 2025 · similarity 0.90)
- BiLLM: Pushing the Limit of Post-Training Quantization for LLMs (ICML 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.