AI Paper Atlas

Research map / Large language models / Language, safety and interpretability

Compressed Sensing for Capability Localization in Large Language Models

Anna Bair, Yixuan Even Xu, Mingjie Sun, J Zico Kolter

ICML 2026 · Large language models › Language, safety and interpretability · approach: Pruning and circuits in LLMs · 0 citations (OpenAlex)

Read the paper

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
ablate top-5 compressed-sensing math headsLlama-3.1-8B · GSM8Kaccuracy change: −40.6 ± 6.8 on GSM8K vs −1.0 ± 0.8 on general benchmarksHurt
ablate top-5 heads on larger vs smaller modelsLlama and Qwen models · task-specific benchmarksaccuracy changeMixed
use stratified instead of Bernoulli measurement matrix (ablation)compressed sensing head identification · GSM8K (Llama-3.1-8B, 100 vs 200 evals)accuracy drop after ablation: −39.5 (CSB, 200 evals) -> −48.4 (CSS, 100 evals)Improved
replace greedy search with compressed sensinghead identification · GSM8K (Llama-3.1-8B)accuracy drop and number of evaluations: −55.4 with 5120 evals -> −48.4 with 100 evalsMixed
replace one-shot greedy with stratified compressed sensinghead identification · Swear (Llama-3.1-8B)accuracy drop: −59.9 -> −85.4Improved
increase number of evaluations from 100 to 300 (ablation)compressed sensing · GSM8K (Llama-3.1-8B)accuracy drop: −40.6 -> −52.4Improved
ablate more than five heads (ablation)math heads · GSM8K and Arithmetic (Llama-3.1-8B)accuracyNo clear effect
transfer identified heads across datasets of same capabilitymath heads · Arithmetic (Llama-3.1-8B)accuracy change: −60.2 ± 4.3 on Arithmetic from GSM8K-identified headsImproved
ablate universal heads (e.g. L1H29)Llama-3.1-8B · Math, code and general benchmarksaccuracy change: L1H29: −81.3 math, −63.4 code, −37.9 generalHurt
apply method to WMDP subtasksLlama-3.1-8B · WMDPaccuracyNo clear effect
compare compressed sensing 300 evals to ProxySPEX 500 evalshead identification · GSM8K (Llama-3.1-8B)accuracy drop: −41.9 vs −41.1No clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy