Research map / Large language models / Language, safety and interpretability
Remarkable Robustness of LLMs: Stages of Inference?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Deletes and swaps adjacent transformer layers at inference across Pythia, GPT-2, Qwen 2.5, LLaMA 3.2 and Phi models, finding robustness in middle layers and proposing a four-stage depth-wise framework of inference (detokenization, feature engineering, prediction ensembling, residual sharpening).
Problem
How LLMs tolerate structural interventions such as dropping or swapping layers at inference, and what depth-dependent computational structure this reveals about how decoder-only transformers perform inference.
Approach
Layer-wise deletion and adjacent-swap interventions show performance is localized to early and final layers while middle layers are robust; these signatures (CKA block structure, local attention, WiC probing, logit lens, prediction/suppression neurons, layer repetition) are synthesized into a four-stage hypothesis of inference.
Key results
- Models retain 72–95% of original top-1 prediction accuracy under layer deletion/swap without fine-tuning.
- Interventions on the first and final layers cause the most degradation, while middle layers are remarkably robust to deletion and swapping.
- Prediction neuron density peaks at roughly 85% of model depth, and repeating late-layer blocks reduces final-layer logit entropy more than repeating early layers.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Delete or swap the first layer during inference (ablation) | GPT-2 and Pythia models · Layer intervention evaluation | top-1 prediction accuracy | Hurt |
| Delete or swap middle layers rather than early or final layers (ablation) | GPT-2 and Pythia models · Layer intervention evaluation | top-1 prediction accuracy | No clear effect |
| Swap adjacent intermediate layers instead of ablating a layer (ablation) | GPT-2 and Pythia models · Layer intervention evaluation | top-1 prediction accuracy | Helped |
| Swap the last layer with the second-to-last layer (ablation) | Some models · Layer intervention evaluation | prediction entropy | Hurt |
| Swap the last layer with the second-to-last layer (ablation) | GPT-2 models · Layer intervention evaluation | top-1 prediction consistency | No clear effect |
| Repeat a block of layers in the latter half of the model (ablation) | Language models · Layer-repetition experiment | prediction entropy | Helped |
| Repeat layers at the last 80-90% of model depth (ablation) | Language models · Downstream benchmarks | benchmark performance | Helped |
| Use prediction-neuron ensemble probes instead of individual-neuron probes | GPT-2 XL · Suffix “-ing” classification | probe accuracy | Helped |
Surprising findings
- Swapping adjacent layers is less harmful than ablating them, suggesting some operations in the forward pass are commutative.
- Ensemble probes on prediction neurons can occasionally exceed the model's own top-1 accuracy on the -ing suffix task.
Limitations
- Stage boundaries are approximate and multiple stages may co-occur within a layer.
- Analysis reflects aggregate trends and does not isolate factors behind model-specific differences.
- Architectural differences (parallel vs sequential attention/MLP) may affect consistency of phenomena across models.
Experimental setup
Datasets: The Pile
Benchmarks: HellaSwag ARC-Easy LAMBADA WiC
Metrics: KL divergence top-1 prediction agreement (relative accuracy) entropy benchmark accuracy probe accuracy
Baselines: zero ablation
Builds on: logit lens iterative inference hypothesis universal neurons centered kernel alignment
Closest papers by meaning
- Brittlebench: Quantifying LLM robustness via prompt sensitivity (NeurIPS 2026 · similarity 0.90)
- Scaling Laws Meet Model Architecture: Toward Inference-Efficient LLMs (ICLR 2026 · similarity 0.90)
- LayerIF: Estimating Layer Quality for Large Language Models using Influence Functions (NeurIPS 2025 · similarity 0.90)
- Sparsity-Guided Holistic Explanation for LLMs with Interpretable Inference-Time Intervention (AAAI 2024 · similarity 0.90)
- Sample Smart, Not Hard: Correctness-First Decoding for Better Reasoning in LLMs (ICLR 2026 · similarity 0.89)
- A Probabilistic Perspective on Unlearning and Alignment for Large Language Models (ICLR 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.