Research map / Large language models / Language, safety and interpretability

LLM Interpretability with Identifiable Temporal-Instantaneous Representation

NeurIPS 2025 · Large language models › Language, safety and interpretability · approach: Linguistic encoding in LLM representations · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes a linear temporal causal representation learning framework (SAE-style linear autoencoder with noise-independence and L1 sparsity terms) that recovers time-delayed and instantaneous concept relations in LLM activations with identifiability guarantees and scales to thousands of dimensions.

Problem

Sparse autoencoders extract LLM features without modeling temporal dependencies, instantaneous relations, or identifiability guarantees, while existing causal representation learning methods do not scale to LLM-sized concept spaces due to Jacobian computation cost.

Approach

Model LLM activations as a linear mixing of latent concepts that follow a linear temporal SEM with lagged (B) and instantaneous (M, strictly lower-triangular) effects; prove identifiability from autocovariance structure plus non-Gaussianity, and estimate with a linear autoencoder, Laplacian independent-noise loss, and L1 sparsity, avoiding costly Jacobians.

Key results

• On synthetic data scaled from 128 to 1024 dimensions with 50M samples, the linear method keeps MCC around 0.9 and scales linearly, while IDOL runs out of memory above 200 dimensions and iCITRIS fails beyond 16.
• On SAEBench, the method reaches sparse probing 0.6736 and absorption 0.0139 (lower is better) versus ReLU SAE 0.6555 and 0.0141, and TopK SAE 0.7141 and 0.0280.
• On semi-synthetic relation recovery, the method scores 19.95 on Legal, 8.63 on XML and 2.66 on Email versus 0.54, 0.94 and 0.74 for SAE+regression.

Measured effects reported in the paper

Surprising findings

• Standard nonparametric CRL identifiability proofs fail in the linear case, since the required Jacobian coefficient matrix has rank at most n whenever the instantaneous network is non-empty.
• The proposed method's reconstruction loss (0.0108) is slightly worse than ReLU and TopK SAEs on reconstruction, while sparse probing lags TopK SAE.

Limitations

• The model is linear, while LLM activation mechanisms are nonlinear; the authors acknowledge this and justify it only heuristically.
• Identifiability requires assumptions (non-Gaussian noise, sparsity on M and B, strictly lower-triangular M, and A5/A6 structure), and real-data recovered structures are only approximately verified.
• Real-LLM evaluation is limited to one small model (pythia-160m) and uses a non-standard relation recovery score without ground truth.

Experimental setup

Datasets: The Pile synthetic latent SEM data

Benchmarks: SAEBench

Metrics: mean correlation coefficient (MCC) relation recovery score reconstruction loss sparse probing accuracy absorption autointerp score

Baselines: ReLU SAE TopK SAE IDOL iCITRIS SAE+regression

Builds on: sparse autoencoders iCITRIS IDOL linear ICA identifiability

Closest papers by meaning

• Sparsity-Guided Holistic Explanation for LLMs with Interpretable Inference-Time Intervention (AAAI 2024 · similarity 0.88)
• Are Sparse Autoencoders Useful? A Case Study in Sparse Probing (ICML 2025 · similarity 0.88)
• From Causal to Concept-Based Representation Learning (NeurIPS 2024 · similarity 0.88)
• Causal Temporal Representation Learning with Nonstationary Sparse Transition (NeurIPS 2024 · similarity 0.88)
• I Have Covered All the Bases Here: Interpreting Reasoning Features in Large Language Models via Sparse Autoencoders (AAAI 2026 · similarity 0.87)
• CASL: Concept-Aligned Sparse Latents for Interpreting Diffusion Models (NeurIPS 2026 · similarity 0.86)