AI Paper Atlas

Research map / Large language models / Language, safety and interpretability

Jacobian Scopes: A Unified Geometric Framework for Token-Level LLM Attributions

Toni Liu, Baran Zadeoğlu, Nicolas Boulle, Raphaël Sarfati, Gurbir Arora, Christopher Earls

NeurIPS 2026 · Large language models › Language, safety and interpretability · approach: Pruning and circuits in LLMs

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Jacobian Scopes, a gradient-based attribution framework that explains full predictive distributions of LLMs at the token level rather than only a single target logit.

Problem

Gradient-based attribution for LLMs has been limited to explaining scalar outputs (the logit of one target token), whereas questions about uncertainty and the spread of plausible continuations concern the full predictive distribution.

Approach

Extends gradient-based token attribution from scalar logits to the full predictive distribution via a unified geometric (Jacobian-based) framework, so that attributions can answer distribution-level questions such as which tokens drive model uncertainty or a broad set of plausible continuations.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
attribute the full predictive distribution (Fisher Scope) instead of a single target logit (Input x Gradient)LLaMA-3.2 1B · LAMBADAAOPC: -1.28 -> -1.32Improved
attribute effective temperature (Temperature Scope) instead of Input x GradientQwen2.5-1.5B · LAMBADAAOPC: -1.72 -> -1.86Improved
attribute effective temperature (Temperature Scope) instead of Input x GradientQwen2.5 3B · LAMBADAAOPC: -1.39 -> -1.55Improved
use Fisher Scope instead of Integrated GradientsGemma-3 4B · IWSLT2017 DE-ENAOPC: -1.34 -> -1.20Hurt
use Semantic Scope instead of Input x GradientLLaMA-3.2 1B · LAMBADAAOPC: -1.28 -> -1.30No clear effect
replace zero-vector ablation with PAD-token ablation (ablation)Jacobian Scopes · LAMBADA and IWSLT2017, six LLMsbest-method rankingNo clear effect
raise Fisher rank from r=1 to r=256 (ablation)Fisher Scope · LAMBADA, LLaMA-3.2 1BAOPCNo clear effect
keep only rank-1 principal Fisher direction (ablation)Fisher Scope · LAMBADA, LLaMA-3.2 1BAOPCImproved
use Fisher Scope instead of Input x GradientGemma-3 4B · LAMBADAAOPC: -1.70 -> -1.81Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy