AI Paper Atlas

Research map / Architectures and efficiency / Pruning and efficient tuning

Understanding Performance Collapse in Layer-Pruned Large Language Models via Decision Representation Transitions

boyu shi, Chang Liu, Chuanbao Gao, Xu Yang, Xin Geng

ICML 2026 · Architectures and efficiency › Pruning and efficient tuning · approach: Layer-wise pruning of LLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
remove layer 17 (late Silent Phase) instead of layer 19 (ablation)Llama3-8B · ARC-Challengeaccuracy: 82.68 -> 72.27Hurt
remove layers 17 and 18 (pre-transition pair) (ablation)Llama3-8B · 7-task averageaccuracy: 75.91 -> 59.24Hurt
remove layer 17 on HellaSwag (ablation)Llama3-8B · HellaSwagaccuracy: 71.92 -> 47.39Hurt
remove layers 17 and 18 on MMLU (ablation)Llama3-8B · MMLUaccuracy: 64.71 -> 41.42Hurt
remove decisive-phase layer 19 (ablation)Llama3-8B · 7-task averageaccuracy: 74.83 -> 75.91No clear effect
remove layer 17 on BoolQ (ablation)Llama3-8B · BoolQaccuracy: 77.89 -> 81.22No clear effect
remove layer 17 on PIQA (ablation)Llama3-8B · PIQAaccuracy: 80.30 -> 80.58No clear effect
use Iterative Pruning instead of ShortGPT before SFT at 12.5% ratioLlama3-8B · 7-task average after SFTaverage accuracy: 64.36 -> 83.76Improved
use Iterative Pruning instead of SLEB at 31.2% ratioLlama3-8B · 7-task average after SFTaverage accuracy: 50.21 -> 84.56Improved
use Iterative Pruning instead of MKA at 21.9% ratioLlama3-8B · 7-task average after SFTaverage accuracy: 59.53 -> 83.49Improved
apply SFT to heavily pruned models (40.6%, 43.8%)Pruned Llama3-8B · ARC-Challenge decision margindecision transitionNo clear effect
apply SFT to moderately pruned model (37.5%)Pruned Llama3-8B · ARC-Challenge decision margindecision marginImproved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy