Research map / Architectures and efficiency / Pruning and efficient tuning
PLoP: Precise LoRA Placement for Efficient Finetuning of Large Models
Soufiane Hayou, Nikhil Ghosh, Bin Yu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| place LoRA by lowest NFN (PLoP) instead of attention-only (ablation) | LoRA on Qwen3-1.7B · MetaMathQA -> GSM8K (SFT) | accuracy: 69.5% -> 75.2% | Improved |
| place LoRA by PLoP instead of attention-only (ablation) | LoRA on Qwen3-0.6B · MetaMathQA -> GSM8K (SFT) | accuracy: 58.6% -> 62.0% | Improved |
| place LoRA by PLoP instead of all modules (ablation) | LoRA on Qwen3-1.7B · MetaMathQA -> GSM8K (SFT) | accuracy: 73.9% -> 75.2% | Improved |
| PLoP instead of MLP placement (ablation) | LoRA on Qwen3-1.7B · MetaMathQA -> GSM8K (SFT) | accuracy: 75.0% -> 75.2% | No clear effect |
| PLoP instead of MLP placement (equal params) (ablation) | LoRA on Qwen3-0.6B · MetaMathQA -> GSM8K (SFT) | accuracy: 63.3% -> 63.8% | Improved |
| choose lowest-NFN modules (PLoP) instead of highest (PLoP^-1) (ablation) | LoRA on Qwen3-1.7B · MetaMathQA -> GSM8K (SFT) | accuracy: 74.6% -> 75.2% | Improved |
| choose lowest-NFN modules (PLoP) instead of highest (PLoP^-1) (ablation) | LoRA with GRPO on Qwen3-1.7B · MetaMathQA -> GSM8K (GRPO) | accuracy: 71.41% -> 74.52% | Improved |
| PLoP instead of MLP at same rank 16 (ablation) | LoRA with GRPO on Qwen3-1.7B · MetaMathQA -> GSM8K (GRPO) | accuracy: 73.61% -> 74.52% | Improved |
| PLoP instead of attention-only at similar parameters (ablation) | LoRA with GRPO on Qwen3-1.7B · MetaMathQA -> GSM8K (GRPO) | accuracy: 72.13% -> 74.52% | Improved |
| raise PLoP rank from 16 to 25 (ablation) | LoRA with GRPO on Qwen3-1.7B · MetaMathQA -> GSM8K (GRPO) | accuracy: 74.52% -> 75.03% | Improved |
| apply GRPO with PLoP LoRA on Gemma3-1B | LoRA with GRPO on Gemma3-1B · MetaMathQA -> GSM8K (GRPO) | accuracy: 29.10% -> 30.52% | No clear effect |
| place LoRA by PLoP vs MLP/Attn on ANLI (ablation) | LoRA on Llama3.2-1B · ANLI | accuracy | No clear effect |
Closest papers by meaning
- $\textit{Trans-LoRA}$: towards data-free Transferable Parameter Efficient Finetuning (NeurIPS 2024 · similarity 0.89)
- LoRA-Pro: Are Low-Rank Adapters Properly Optimized? (ICLR 2025 · similarity 0.89)
- LoRA-GA: Low-Rank Adaptation with Gradient Approximation (NeurIPS 2024 · similarity 0.88)
- Sparse High Rank Adapters (NeurIPS 2024 · similarity 0.88)
- SeedLoRA: A Fusion Approach to Efficient LLM Fine-Tuning (ICML 2025 · similarity 0.88)
- LoRA+: Efficient Low Rank Adaptation of Large Models (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
