Research map / Large language models / Language, safety and interpretability

A Unifying Perspective on Language Model Interpretability

NeurIPS 2026 · Large language models › Language, safety and interpretability · approach: Linguistic encoding in LLM representations

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's abstract only. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes viewing language model representations through Tensor Product Representations (TPRs) as a unifying framework that could connect diverse interpretability methods.

Problem

Interpretability methods for language models yield insights that remain isolated from one another; there is no common underlying structure that explains why these diverse interpretations arise.

Approach

Uses Tensor Product Representations (Smolensky 1990) as a candidate unifying structural model of language model internals, so that different interpretability findings can be understood as consequences of one underlying representational format.

Experimental setup

Builds on: Tensor Product Representations (Smolensky 1990)

Closest papers by meaning

• Towards Universality: Studying Mechanistic Similarity Across Language Model Architectures (ICLR 2025 · similarity 0.88)
• LLM Layers Immediately Correct Each Other (NeurIPS 2025 · similarity 0.87)
• Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Models (ICML 2024 · similarity 0.87)
• Invariant Features in Language Models: Geometric Characterization and Model Attribution (NeurIPS 2026 · similarity 0.87)
• Towards Interpretability Without Sacrifice: Faithful Dense Layer Decomposition with Mixture of Decoders (NeurIPS 2025 · similarity 0.87)
• Extracting Interpretable Task-Specific Circuits from Large Language Models for Faster Inference (AAAI 2025 · similarity 0.87)