Research map / Architectures and efficiency / RNNs and normalization
HyperDAS: Towards Automating Mechanistic Interpretability with Hypernetworks
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes HyperDAS, a transformer hypernetwork that automatically selects token positions and learns Householder-parameterized feature subspaces for interchange interventions, achieving state-of-the-art disentanglement on RAVEL with Llama3-8B.
Problem
Distributed alignment search for localizing concept features in LM hidden states requires brute-force search over layers and token positions, and heuristics for token selection are often wrong.
Approach
A transformer hypernetwork conditioned on a natural-language concept description cross-attends to base and counterfactual model runs to produce a soft token-alignment matrix and a Householder-transformed orthogonal subspace, trained end-to-end with a RAVEL loss plus a sparsity loss and then snapped to discrete choices at evaluation.
Key results
- HyperDAS achieves state-of-the-art RAVEL Disentangle performance on Llama3-8B, outperforming MDAS across all entity splits at layer 15.
- Training one HyperDAS model across all RAVEL domains is only 4.0% worse than per-domain models while still beating MDAS.
- HyperDAS requires about 2.4x the training compute of MDAS per epoch (468,923 vs 193,833 TeraFLOPs) but uses less memory when trained on all 23 attributes (68GB vs 110.3GB).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use HyperDAS instead of MDAS | RAVEL, city country attribute training · RAVEL, city country attribute, one epoch | training compute: 468,923 vs 193,833 TeraFLOPs | Hurt |
| use HyperDAS instead of MDAS | RAVEL, all 23 attributes · RAVEL, all 23 attributes | memory usage: 110.3GB vs 68GB | Helped |
| train one HyperDAS model across all domains instead of a separate model per entity type (ablation) | HyperDAS · RAVEL, five entity domains | Disentangle: 4.0% | Mixed |
| initialize HyperDAS from pretrained Llama3-8B parameters instead of from scratch (ablation) | HyperDAS · RAVEL, city | training loss | No clear effect |
| remove sparsity loss (ablation) | HyperDAS · RAVEL, Occupation | Disentangle with weighted interventions: ≈ 94.0% | No clear effect |
| remove sparsity loss (ablation) | HyperDAS · RAVEL, Occupation, one-to-one evaluation | Disentangle | Hurt |
| use excessive sparsity loss (ablation) | HyperDAS · RAVEL, Occupation, one-to-one evaluation | Disentangle | Hurt |
| use excessive sparsity loss (ablation) | HyperDAS · RAVEL, Occupation, weighted interventions | Disentangle: ≈ 94.0% | No clear effect |
| use symmetric localization instead of asymmetric localization (ablation) | HyperDAS · RAVEL | Disentangle | No clear effect |
Surprising findings
- Without a sparsity loss, the hypernetwork aligns the final entity token to many base tokens, which fails under one-to-one evaluation.
- The hypernetwork can learn a trivial shortcut of intervening only when the base attribute differs from the target attribute unless the base attribute is masked.
Limitations
- Only works if the target concept is mediated by linear features; non-linear mediators are not handled.
- Higher training compute than MDAS per epoch.
- Risk of false-positive interpretations from a powerful supervised hypernetwork injecting information or steering the model.
Experimental setup
Datasets: RAVEL
Benchmarks: RAVEL
Metrics: Cause score Iso score Disentangle score
Baselines: DAS MDAS sparse autoencoder
Builds on: distributed alignment search interchange intervention RAVEL MDAS
Closest papers by meaning
- Hyperbolic Concept Bottleneck Models (NeurIPS 2026 · similarity 0.85)
- Predictive Concept Decoders: Training Scalable End-to-End Interpretability Assistants (NeurIPS 2026 · similarity 0.85)
- Foundation Models Secretly Understand Neural Network Weights: Enhancing Hypernetwork Architectures with Foundation Models (ICLR 2025 · similarity 0.85)
- HyperLogic: Enhancing Diversity and Accuracy in Rule Learning with HyperNets (NeurIPS 2024 · similarity 0.84)
- Mechanistic Permutability: Match Features Across Layers (ICLR 2025 · similarity 0.84)
- Magnitude Invariant Parametrizations Improve Hypernetwork Learning (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.