Research map / Large language models / Instruction and fine-tuning
Parameter-Efficient Fine-Tuning Methods for Pretrained Language Models: A Critical Review and Assessment
Lingling Xu, Haoran Xie, S. Joe Qin, Xiaohui Tao, Fu Lee Wang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace full fine-tuning with sequential adapter | RoBERTa-base · GLUE | average: 84.00 -> 85.39 | Improved |
| replace full fine-tuning with sequential adapter | RoBERTa-large · GLUE | average: 86.98 -> 88.58 | Improved |
| replace full fine-tuning with prompt-tuning (0.61M params) | RoBERTa-base · GLUE | average: 84.00 -> 74.56 | Hurt |
| replace full fine-tuning with prefix-tuning | RoBERTa-base · GLUE | average: 84.00 -> 80.89 | Hurt |
| replace full fine-tuning with ProPELT_Adapter (about 1.50% params) | RoBERTa-large · GLUE | average: 86.98 -> 88.70 | Improved |
| replace full fine-tuning with LoRA | RoBERTa-base · GLUE | average: 84.00 -> 84.33 | Improved |
| replace full fine-tuning with AdaLoRA | RoBERTa-base · GLUE | average: 84.00 -> 83.61 | Hurt |
| replace full fine-tuning with Child-Tuning_D | RoBERTa-large · GLUE MNLI | accuracy: 89.42 -> 35.45 | Hurt |
| add MAM adapter (parallel adapter plus prefix-tuning) in place of prefix-tuning | RoBERTa-base · GLUE | average: 80.89 -> 84.26 | Improved |
| replace full fine-tuning with (IA)3 | T5-base · WMT16 En-Ro | BLEU: 0.16 higher with 0.03% trainable parameters | Improved |
| replace full fine-tuning with LoRA | T5-base · WMT16 En-Ro | BLEU: 0.36 higher with 0.39% trainable parameters | Improved |
| replace full fine-tuning with LoRA | T5-large · WMT16 En-Ro | BLEU: 0.01 lower with 0.32% trainable parameters | No clear effect |
Closest papers by meaning
- Linearization Explains Fine-Tuning in Large Language Models (NeurIPS 2025 · similarity 0.91)
- Preference-Oriented Supervised Fine-Tuning: Favoring Target Model over Aligned Large Language Models (AAAI 2025 · similarity 0.90)
- Interweaving Memories of a Siamese Large Language Model (AAAI 2025 · similarity 0.90)
- Get more for less: Principled Data Selection for Warming Up Fine-Tuning in LLMs (ICLR 2024 · similarity 0.89)
- Bilevel ZOFO: Efficient LLM Fine-Tuning and Meta-Training (NeurIPS 2025 · similarity 0.89)
- Parameter-Efficient Fine-Tuning of State Space Models (ICML 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
