Research map / Large language models / Language, safety and interpretability

Patchscopes: A Unifying Framework for Inspecting Hidden Representations of Language Models

ICML 2024 · Large language models › Language, safety and interpretability · approach: Linguistic encoding in LLM representations · 2 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes Patchscopes, a framework that decodes hidden representations by patching them into a target inference pass of an LLM (same or larger model) with a prompt designed to elicit the desired information, unifying logit lens, tuned lens, future lens, and causal tracing as special cases.

Problem

Existing interpretability methods for LLM hidden representations (probing, vocabulary projection, causal interventions) are limited: probes need supervised training over predefined classes, vocabulary projections fail in early layers, and none give expressive natural-language explanations.

Approach

Decode a representation by patching it into a separate inference pass on a designed target prompt (possibly on a different or larger model), so the model itself verbalizes the information; this unifies prior inspection methods as configurations and enables training-free, expressive, layer-robust inspection.

Key results

• Few-shot token identity Patchscope outperforms logit lens and tuned lens for next-token prediction from layer 10 onward, with gains up to 98% in layers 18-22.
• Zero-shot feature extraction Patchscope significantly beats logistic regression probing on 6 of 12 tasks (p<1e-5) without training data.
• On multi-hop error correction, the CoT Patchscope reaches 50% accuracy versus 35.71% for chain-of-thought and 19.57% for vanilla generation (Vicuna-13B, 46 samples).

Measured effects reported in the paper

Surprising findings

• Patching representations from a smaller model (Vicuna-7B) into a larger one (Vicuna-13B) improves entity-resolution inspection, while for Pythia the smaller model outperformed the larger one, so cross-model patching was less effective there.
• Feature extraction Patchscope underperforms probing in later layers, attributed to representations shifting toward next-token prediction.

Limitations

• Evaluation focuses on autoregressive Transformer LLMs; generality to other architectures untested.
• Placeholder token contamination can degrade results when patching later layers in multi-token generation.
• Multi-hop evaluation uses a small set (46 samples) and manually exploits known query structure.

Experimental setup

Datasets: The Pile PopQA WikiText-103 Commonsense and factual relation dataset (Hernandez et al. 2023)

Benchmarks: The Pile evaluation set Commonsense and factual relation tasks (5 commonsense, 7 factual) PopQA Multi-hop factual and commonsense queries (46 evaluated)

Metrics: Precision@1 Surprisal Accuracy RougeL Rouge1 Sentence-BERT similarity t-test with Bonferroni correction

Baselines: Logit Lens Tuned Lens Logistic regression probe Chain-of-thought prompting Vanilla generation

Builds on: logit lens tuned lens future lens causal tracing activation patching

Closest papers by meaning

• Training Language Models to Explain Their Own Computations (NeurIPS 2026 · similarity 0.89)
• Potemkin Understanding in Large Language Models (ICML 2025 · similarity 0.89)
• Sparsity-Guided Holistic Explanation for LLMs with Interpretable Inference-Time Intervention (AAAI 2024 · similarity 0.89)
• Automated Assessment of Fidelity and Interpretability: An Evaluation Framework for Large Language Models’ Explanations (Student Abstract) (AAAI 2024 · similarity 0.88)
• metabench - A Sparse Benchmark of Reasoning and Knowledge in Large Language Models (ICLR 2025 · similarity 0.88)
• Calibrating Reasoning in Language Models with Internal Consistency (NeurIPS 2024 · similarity 0.88)