Research map / Architectures and efficiency / Attention and transformers
Progressive Inference: Explaining Decoder-Only Sequence Classification Models Using Intermediate Predictions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Progressive Inference, which reuses intermediate predictions of decoder-only classifiers as masked-input predictions to compute input attributions, via a zero-cost single-pass variant (SP-PI) and a Kernel SHAP-connected multi-pass variant (MP-PI).
Problem
Explaining predictions of decoder-only transformer sequence classifiers with input attributions that are faithful and affordable; prior attribution methods are either expensive (perturbation/SHAP) or yield low-quality attributions for Transformers.
Approach
Due to causal masked attention, the classification head's prediction at position i depends only on tokens up to i, so intermediate predictions approximate the model's output on prefix-masked inputs. SP-PI takes successive differences of these predictions (zero extra cost); MP-PI samples random masks over multiple passes and solves a Kernel SHAP-style weighted regression with an optimized mask dist
Key results
- MP-PI achieves up to 10.3% AUC improvement over the best prior work in the activation study (e.g., 0.946 on SST-2 vs 0.899 for Kernel SHAP).
- MP-PI reduces inverse-activation AUC by up to 57.5% relative to the best prior work (e.g., 0.214 vs 0.337 on Twitter-Emotion).
- SP-PI, at zero extra cost, is the best method among cost<=1x methods on most datasets in both studies.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MP-PI instead of Kernel SHAP | Attribution method · IMDB | activation AUC: 0.959 -> 0.97 | Helped |
| Use MP-PI instead of Kernel SHAP | Attribution method · SST-2 | activation AUC: 0.899 -> 0.946 | Helped |
| Use MP-PI instead of Kernel SHAP | Attribution method · AG-News | activation AUC: 0.778 -> 0.867 | Helped |
| Use MP-PI instead of Kernel SHAP | Attribution method · Twitter-Fin | activation AUC: 0.821 -> 0.847 | Helped |
| Use MP-PI instead of Kernel SHAP | Attribution method · Twitter-Sentiment | activation AUC: 0.816 -> 0.887 | Helped |
| Use MP-PI instead of Kernel SHAP | Attribution method · Twitter-Emotion | activation AUC: 0.871 -> 0.929 | Helped |
| Use MP-PI instead of Kernel SHAP | Attribution method · TrueTeacher NLI | activation AUC: 0.811 -> 0.921 | Helped |
| Use MP-PI instead of Kernel SHAP | Attribution method · IMDB | inverse activation AUC: 0.688 -> 0.613 | Helped |
| Use MP-PI instead of Kernel SHAP | Attribution method · SST-2 | inverse activation AUC: 0.533 -> 0.431 | Helped |
| Use MP-PI instead of Kernel SHAP | Attribution method · AG-News | inverse activation AUC: 0.734 -> 0.596 | Helped |
| Use MP-PI instead of Kernel SHAP | Attribution method · Twitter-Fin | inverse activation AUC: 0.709 -> 0.684 | Helped |
| Use MP-PI instead of Kernel SHAP | Attribution method · Twitter-Sentiment | inverse activation AUC: 0.348 -> 0.273 | Helped |
Surprising findings
- Taking the l2 norm improved AUC for Inp x Grad and LRP baselines, which the authors used to strengthen prior work.
- The optimized masking distribution P' gives only marginal gains over directly sampling from the Shapley distribution (differences often under 0.01 AUC).
Limitations
- Masked inputs using a '...' mask token may not be valid inputs for the model, so attributions assume masked inputs are meaningful.
- MP-PI differs from true SHAP values because samples are correlated due to masked attention and intermediate predictions may not exactly match masked-input predictions.
Experimental setup
Datasets: IMDB SST-2 AG-News Twitter-Finance Twitter-Sentiment Twitter-Emotion TrueTeacher NLI
Benchmarks: IMDB SST-2 AG-News Twitter-Finance Twitter-Sentiment Twitter-Emotion TrueTeacher NLI
Metrics: AUC of activation study (higher better) AUC of inverse activation study (lower better) cosine similarity to Kernel SHAP
Baselines: Random Attention-last Generic attention-model explainability (GAE) Gradient x Input LRP for Transformers Integrated Gradients Kernel SHAP
Builds on: Kernel SHAP Integrated Gradients LoRA Layer-wise Relevance Propagation
Closest papers by meaning
- Towards Understanding Self-Pretraining for Sequence Classification (NeurIPS 2026 · similarity 0.85)
- How Transformers Learn Structured Data: Insights From Hierarchical Filtering (ICML 2025 · similarity 0.85)
- MaskPro: Linear-Space Probabilistic Learning for Strict (N:M)-Sparsity on LLMs (ICLR 2026 · similarity 0.85)
- How Transformers Learn Regular Language Recognition: A Theoretical Study on Training Dynamics and Implicit Bias (ICML 2025 · similarity 0.84)
- On the Generalization Ability of Next-Token-Prediction Pretraining (ICML 2025 · similarity 0.84)
- Linear Attention for Efficient Bidirectional Sequence Modeling (NeurIPS 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.