AI Paper Atlas

Research map / Large language models / Language, safety and interpretability

First is Not Really Better Than Last: Evaluating Layer Choice and Aggregation Strategies in Language Model Data Influence Estimation

Dmytro Vitel, Anshuman Chhabra

ICLR 2026 · Large language models › Language, safety and interpretability · approach: Pruning and circuits in LLMs · 1 citations (OpenAlex)

Read the paper

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
use Vote instead of Mean aggregation of layer scoresDataInf · Noise detection vs accuracy correlation (Mistral 7B)Spearman correlation of NDR: 0.5 -> 0.9Improved
use Vote instead of Mean aggregationTracIn · Mistral 7Blayer rank of classification head: 12 -> 1Improved
use Vote instead of Mean aggregationTracIn · Roberta-Largelayer rank of classification head: 10 -> 2Improved
use Vote instead of Mean aggregationDataInf layers 00-07 · Llama/GLUE filteringwin rate: 0.84Improved
use Vote aggregation instead of MeanCosine · Mistral 7B GLUEaccuracy: degradedHurt
use Vote instead of Mean aggregationDataInf layers 18-23 · Roberta-Large COLAtest accuracy: 83.3 -> 79.1Hurt
use Vote instead of Mean aggregationDataInf layers 18-23 · Roberta-Large QNLItest accuracy: 87.4 -> 87.7Improved
filter with influence scores versus randomDataInf Mean layers 18-23 · Roberta-Large RTEtest accuracy: 57.8 -> 60.5Improved
select cancellation-based low-C layers as influence estimatorgradient cancellation C · Roberta-Large/Llama/MistralSpearman correlation with accuracy: 0.2No clear effect
use attention layers rather than classification headTracIn · Mistral 7Bwin rate: .20 -> .61Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy