AI Paper Atlas

Research map / Large language models / Language, safety and interpretability

LLMs Process Lists With General Filter Heads

Arnab Sen Sharma, Giordano Rogers, Natalie Shapira, David Bau

ICLR 2026 · Large language models › Language, safety and interpretability · approach: Reasoning mechanisms and failure modes

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
ablate filter heads (attention restricted to BOS) (ablation)Llama-70B · Select* taskstask accuracyHurt
ablate filter heads (ablation)Llama-70B · Counting and CheckExistencetask accuracyNo clear effect
ablate is_match flag in item latents with question placed before items (ablation)Llama-70B · SelectOne objectsaccuracy: 46.09% with ablation (question before)Hurt
ablate is_match flag with question placed after items (ablation)Llama-70B · SelectOne objectsaccuracy: 96.06% with ablation (question after)No clear effect
place question before the items instead of afterfilter heads · SelectOne objectscausality: near zeroHurt
add priming prefixes recalling relevant informationfilter heads · person-nationality and landmark-countrycausality: approximately 10 pointsImproved
transfer filter heads to non-semantic filtering (rhyme)filter heads · rhyming wordscausalityHurt
increase distractors in collection up to 7filter heads · SelectOne objectscausality: close to 0.8No clear effect
use learned sparse masks over heads instead of single-head patching (ablation)filter head localization · SelectOne objectscollective causalityImproved
swap key states of target and other item in addition to query patch (ablation)filter heads · SelectOne objectscausality: 0.783 (432/552)Improved
transfer filter heads across Select* tasksfilter heads · SelectOne, MCQ, First, Lastcausality: at least 70%Improved
transfer Select* filter heads to Countingfilter heads · CountingcausalityHurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy