AI Paper Atlas

Research map / Architectures and efficiency / Pruning and efficient tuning

SliMOO: Interpretable Multi-Objective Evolutionary Search for LLM Depth Pruning

Guanchen Li, Yixing Xu, Xuanwu Yin, Dong Li, Emad Barsoum

NeurIPS 2026 · Architectures and efficiency › Pruning and efficient tuning · approach: Layer-wise pruning of LLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SliMOO, an interpretable multi-objective evolutionary search for LLM depth pruning that accounts for task-dependent layer contributions.

Problem

Depth pruning of LLMs to reduce serving cost, where existing methods ignore that transformer layers contribute differently across tasks and tend to optimize for a single average objective.

Approach

Treats depth pruning as multi-objective search over which layers to remove, using evolutionary search so that task-specific performance trade-offs are preserved rather than collapsed into one objective; the abstract describes the approach as interpretable.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
Use multi-objective Pareto search with TLIP instead of EvoPressSliMOO · Llama-3.1-70B 25% layer pruningMath avg: 24.33 -> 34.40Improved
Use SliMOO instead of EvoPress, language perplexitySliMOO · Llama-3.1-70B 25% layer pruningLanguage perplexity avg: 9.88 -> 9.89No clear effect
Add TLIP prior guidance on top of NSGA-II (ablation)SliMOO · Llama-3.1-70B 25% layer pruningCode avg: 36.02 -> 39.14Improved
Use NSGA-II instead of scalarized mean objective (ablation)SliMOO · Llama-3.1-70B 25% layer pruningMath avg: 29.14 -> 32.67Improved
Use NSGA-II instead of hypervolume-only selection (ablation)SliMOO · Llama-3.1-70B 25% layer pruningMath avg: 30.77 -> 32.67Improved
Use adaptive temperature instead of fixed tau=1 (ablation)SliMOO · Llama-3.1-70B 25% layer pruningMath avg: 32.61 -> 34.40Improved
Increase generations from 20 to 30 (ablation)SliMOO · Llama-3.1-70B 25% layer pruningMath avg: 29.49 -> 34.40Improved
Increase offspring from 64 to 128 (ablation)SliMOO · Llama-3.1-70B 25% layer pruningKnowledge avg: 66.90 -> 65.77Mixed
Halve calibration budget from 64K to 32K tokens (ablation)SliMOO · Llama-3.1-70B 25% layer pruningAll domains: 34.40 -> 34.40No clear effect
Reduce calibration to 8K tokens per domain (ablation)SliMOO · Llama-3.1-70B 25% layer pruningMath avg: 34.40 -> 29.04Hurt
Use math-dominant calibration allocation (ablation)SliMOO · Llama-3.1-70B 25% layer pruningMath avg: 34.40 -> 35.16Mixed
Rerun baselines with mixed multi-domain calibration (EvoPress-mix) (ablation)EvoPress · Llama-3.1-70B 25% layer pruningMath avg: 24.33 -> 27.89Improved

Earlier methods this paper compares with

Named baselines from earlier accepted papers in this paper's results tables: better than 1 of 1 on most shared results.

Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy