AI Paper Atlas

Research map / Architectures and efficiency / RNNs and normalization

Negative Pre-activations Differentiate Syntax

Linghao Kong, Angelina Ning, Micah Adler, Nir N Shavit

ICLR 2026 · Architectures and efficiency › RNNs and normalization · approach: Mechanistic circuit interpretability

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
zero only negative pre-activations of top 1% Wasserstein neurons per layer (ablation)Llama 3.1 8B · TSEaccuracy: 84.4 -> 55.2Hurt
zero only negative pre-activations of top 1% Wasserstein neurons per layer (ablation)Llama 3.1 8B · BLiMPaccuracy: 81.2 -> 77.5Hurt
zero negative pre-activations of top 1% WD neurons in Qwen3 8B Base (ablation)Qwen3 8B Base · TSEaccuracy: 84.8 -> 78.6Hurt
perplexity-matched ablation of many low-WD neurons instead of top Wasserstein neurons (ablation)Llama 3.1 8B · BLiMP and TSEaccuracy: BLiMP 81.2 -> 81.5; TSE 84.4 -> 85.9No clear effect
ablate 1% random neurons' negative pre-activations (ablation)Llama 3.1 8B · BLiMP and TSEaccuracy: TSE 84.4 -> 84.1No clear effect
perplexity-matched low-WD ablation vs Wasserstein ablation on non-grammatical benchmarks (ablation)Llama 3.1 8B · ARC, BoolQ, HellaSwag, PIQA, SciQ, WinoGrande, TruthfulQAaccuracy: approx 11% vs approx 4% absolute error increaseHurt
Wasserstein ablation on TruthfulQA (outlier) (ablation)Llama 3.1 8B · TruthfulQAaccuracy: 45.2 -> 39.8 vs perplexity-matched 42.9Hurt
ablate early layers (first four) Wasserstein neurons (ablation)Llama 3.1 8B · TSE NPI licensingerror: 20% increase in errorHurt
cumulative layer-group ablations of Wasserstein neurons (ablation)Llama 3.1 8B · BLiMP and TSEerrorHurt
replace SiLU positive branch with ReLU in top-WD neurons (ablation)Llama 3.1 8B · WikiText 2 validationperplexityNo clear effect
flip sign of negative post-activations of top 2% WD neurons (ablation)Llama 3.1 8B · WikiText 2 validationperplexity: 6.50 -> 2277Hurt
select neurons by MD instead of WD (ablation)Llama 3.1 8B · BLiMP and TSEaccuracyNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy