Research map / Architectures and efficiency / Pruning and efficient tuning
LoRA: Low-Rank Adaptation of Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LoRA instead of full fine-tuning | GPT-3 175B · WikiSQL validation | accuracy (%): 73.8 -> 74.0 | No clear effect |
| Use LoRA instead of full fine-tuning | GPT-3 175B · MultiNLI-m validation | accuracy (%): 89.5 -> 91.7 | Helped |
| Use LoRA instead of full fine-tuning | GPT-3 175B · SAMSum validation | ROUGE-1: 52.0 -> 53.8 | Helped |
| Use LoRA instead of full fine-tuning | GPT-2 medium · E2E NLG Challenge | BLEU: 68.2 -> 70.4 | Helped |
| Use LoRA instead of full fine-tuning | GPT-2 large · E2E NLG Challenge | BLEU: 68.5 -> 70.4 | Helped |
| Use LoRA instead of full fine-tuning | RoBERTa base · GLUE | average score: 86.4 -> 87.2 | Helped |
| Use LoRA instead of full fine-tuning | RoBERTa large · GLUE | average score: 88.9 -> 89.0 | No clear effect |
| Use LoRA instead of full fine-tuning | DeBERTa XXL · GLUE | average score: 91.1 -> 91.3 | No clear effect |
| Use LoRA instead of fine-tuning | GPT-3 175B · MNLI-m-100 | accuracy (%): 60.2 -> 63.8 | Helped |
| Adapt Wq and Wv rather than Wq alone under the same parameter budget (ablation) | GPT-3 LoRA · WikiSQL validation | accuracy (%): 70.4 -> 73.7 | Helped |
| Set rank to 1 for both Wq and Wv (ablation) | GPT-3 LoRA · WikiSQL validation | accuracy (%): 73.7 -> 73.4 | No clear effect |
| Add prefix-embedding tuning to LoRA (ablation) | GPT-3 LoRA · WikiSQL validation | accuracy (%): 73.8 -> 75.0 | Helped |
Closest papers by meaning
- Uni-LoRA: One Vector is All You Need (NeurIPS 2025 · similarity 0.93)
- RefLoRA: Refactored Low-Rank Adaptation for Efficient Fine-Tuning of Large Models (NeurIPS 2025 · similarity 0.93)
- Beyond Higher Rank: Token-wise Input-Output Projections for Efficient Low-Rank Adaptation (NeurIPS 2025 · similarity 0.93)
- Make LoRA Great Again: Boosting LoRA with Adaptive Singular Values and Mixture-of-Experts Optimization Alignment (ICML 2025 · similarity 0.93)
- Train Small, Infer Large: Memory-Efficient LoRA Training for Large Language Models (ICLR 2025 · similarity 0.93)
- The Expressive Power of Low-Rank Adaptation (ICLR 2024 · similarity 0.93)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.