Research map / Large language models / Language, safety and interpretability
SPEX: Scaling Feature Interaction Explanations for LLMs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SPEX, a model-agnostic, spectral-based algorithm for scalable feature-interaction attribution in LLM and VLM outputs, scaling to much longer inputs than prior interaction-importance methods.
Problem
Post-hoc explanations like SHAP give marginal attributions, and existing interaction-importance extensions only scale to about 20 input features, which is far too few for LLM prompts and contexts.
Approach
Uses spectral (sparse Fourier-style) recovery of interaction effects from a modest number of ablated input samples, so the number of model queries grows far more slowly with input length than exhaustive interaction enumeration; the method attributes pairwise and higher-order interactions among prompt/context features.
Key results
- SPEX scales interaction attribution to input lengths far beyond the roughly 20-feature limit of prior interaction-importance methods (abstract claim).
Limitations
- Only the abstract and figure source are available; detailed experimental results are not visible in the provided text.
Experimental setup
Baselines: SHAP marginal attribution SHAP-IQ
Builds on: SHAP SHAP-IQ
Closest papers by meaning
- Sparsity-Guided Holistic Explanation for LLMs with Interpretable Inference-Time Intervention (AAAI 2024 · similarity 0.86)
- I Have Covered All the Bases Here: Interpreting Reasoning Features in Large Language Models via Sparse Autoencoders (AAAI 2026 · similarity 0.86)
- LLM Maybe LongLM: SelfExtend LLM Context Window Without Tuning (ICML 2024 · similarity 0.85)
- Extracting Interpretable Task-Specific Circuits from Large Language Models for Faster Inference (AAAI 2025 · similarity 0.85)
- LoFiT: Localized Fine-tuning on LLM Representations (NeurIPS 2024 · similarity 0.85)
- LLM-wrapper: Black-Box Semantic-Aware Adaptation of Vision-Language Models for Referring Expression Comprehension (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.