AI Paper Atlas

Research map / Large language models / Instruction and fine-tuning

Reassessing Layer Pruning in LLMs: New Insights and Methods

Yao Lu, Hao Cheng, Yujie Fang, Zeyu Wang, Jiaheng Wei, Dongwei Xu, Qi Xuan, Zhaowei Zhu

ICLR 2026 · Large language models › Instruction and fine-tuning · approach: Efficient LLM fine-tuning systems · 32 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use reverse-order layer selection instead of PPL (ablation)LoRA-tuned pruned LLMs · 25% pruning, 4 modelsAvg accuracy: outperforms the second-best PPL metric by 5.30%Improved
use reverse-order instead of Taylor on Llama-3.1-8B-It (ablation)pruned Llama-3.1-8B-It · 25% pruning, LoRAAvg accuracy: 0.4796 -> 0.5268Improved
use reverse-order instead of Taylor on Gemma2-2B-It (ablation)pruned Gemma2-2B-It · 25% pruning, LoRAAvg accuracy: 0.5073 -> 0.5032No clear effect
use partial-layer fine-tuning (lm_head+last three layers) instead of LoRA (ablation)pruned Llama-3.1-8B-It · 25% reverse-order pruningAvg accuracy: 0.5268 -> 0.5807Improved
use partial-layer fine-tuning (lm_head+last three layers) instead of LoRA (ablation)pruned Qwen1.5-7B · 25% reverse-order pruningAvg accuracy: 0.5037 -> 0.5445Improved
use partial-layer fine-tuning (lm_head+last two layers) instead of LoRA (ablation)pruned Vicuna-7B-v1.5 · 25% reverse-order pruningAvg accuracy: 0.4909 -> 0.5060Improved
use QLoRA instead of LoRA (ablation)pruned Llama-3.1-8B-It · 25% reverse-order pruningAvg accuracy: 0.5268 -> 0.5171Hurt
fine-tune more last layers (lm_head only to lm_head+last three layers) (ablation)pruned Llama-3.1-8B-It · 25% reverse-order pruningAvg accuracy: 0.5599 -> 0.5807Improved
use partial-layer fine-tuning instead of LoRA on the unpruned model (ablation)Llama-3.1-8B-It dense · Full-model fine-tuningAvg accuracy: 0.6354 -> 0.6337No clear effect
use iterative pruning (1:4:8) instead of one-shot with LoRA (ablation)pruned Llama-3.1-8B-It, reverse-order · 25% pruningAvg accuracy: 0.5268 -> 0.5455Improved
use iterative pruning (1:1:8) instead of one-shot with Taylor and partial-layer fine-tuning (ablation)pruned Llama-3.1-8B-It, Taylor · 25% pruningAvg accuracy: 0.5701 -> 0.3753Hurt
use iterative pruning (1:1:6) instead of one-shot with Taylor and LoRA (ablation)pruned Gemma2-2B-It · 25% pruningAvg accuracy: 0.5073 -> 0.4947Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy