Research map / Architectures and efficiency / RNNs and normalization

Predictive Concept Decoders: Training Scalable End-to-End Interpretability Assistants

NeurIPS 2026 · Architectures and efficiency › RNNs and normalization · approach: Neural-symbolic reasoning · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Trains a sparse-concept encoder and LoRA-adapted LM decoder end-to-end (pretrained on FineWeb next-token prediction, finetuned on SynthSys belief QA) to make Llama-3.1-8B activations legible via natural-language answers.

Problem

Interpreting neural activations and surfacing behaviors a model cannot self-report is hard; existing scalable interpretability relies on hand-designed agents that are bottlenecked by off-the-shelf model capabilities.

Approach

Turn behavior prediction from activations into an end-to-end objective: a TopK sparse concept encoder (communication bottleneck) feeds a LatentQA-style decoder, jointly pretrained with next-token prediction on web text, with an auxiliary loss that revives dead concepts.

Key results

• PCD (72M tokens, k=16) reaches a 50% jailbreak awareness rate on the Dream template while all other methods are below 30%.
• Auto-interp and recall metrics improve with pretraining data up to about 100M tokens, after which they plateau; the auxiliary loss keeps over 90% of concepts active at 72M tokens.
• At 36M+ pretraining tokens, PCDs match LatentQA on SynthSys multiple-choice accuracy, while a frozen SAE encoder performs poorly.

Surprising findings

• Simply asking the subject model about hint usage fails to elicit it, whereas PCD and LatentQA reveal it.
• KL-based SAE objectives plateau earlier and do worse on recall with more data, unlike L2 SAEs that benefit from larger k.

Limitations

• Auto-interp and recall plateau past about 100M tokens, and KL-based objectives saturate earlier.
• Performance at k=16 is harmed by the bottleneck on complex tasks such as jailbreak awareness, and k changes at test time move inputs out of distribution.
• Case-study findings like legal-liability motivations cannot be directly verified.

Experimental setup

Datasets: FineWeb SynthSys(8B)

Benchmarks: SynthSys FineWeb suffix prediction jailbreak awareness (3 Words, Distractors, Dream) secret hint task (prime numbers) introspection concept injection

Metrics: auto-interp Pearson correlation recall classification accuracy multiple-choice accuracy awareness rate hint-usage rate directly relevant rate

Baselines: LatentQA TopK SAE KL SAE prompting baselines (first-person and third-person)

Builds on: LatentQA TopK sparse autoencoders Concept Bottleneck Models LoRA auto-interp

Closest papers by meaning

• Training Language Models to Explain Their Own Computations (NeurIPS 2026 · similarity 0.88)
• Explainable Neural Networks with Guarantee: A Sparse Estimation Approach (AAAI 2025 · similarity 0.87)
• Enhancing Performance of Explainable AI Models with Constrained Concept Refinement (ICML 2025 · similarity 0.87)
• Explanation Bottleneck Models (AAAI 2025 · similarity 0.87)
• Explaining, Fast and Slow: Abstraction and Refinement of Provable Explanations (ICML 2025 · similarity 0.87)
• ActivationReasoning: Logical Reasoning in Latent Activation Spaces (ICLR 2026 · similarity 0.87)