Research map / Architectures and efficiency / Attention and transformers
DETAIL: Task DEmonsTration Attribution for Interpretable In-context Learning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DETAIL, an influence-function attribution method for in-context demonstrations that treats the transformer as an internal kernelized ridge regressor, with random projection for speed, and applies it to demonstration reordering and curation.
Problem
Attributing which in-context learning demonstrations help or harm a transformer's prediction, in a way that is fast, sensitive to ordering, demonstration-level, and does not require parameter updates.
Approach
Reformulates the influence function on the kernelized ridge regression that transformers implicitly perform during ICL, using hidden states of demonstrations as features; computes self-influence by reusing demonstration embeddings (one forward pass) and reduces dimension by random Johnson-Lindenstrauss projection for speed.
Key results
- On demonstration curation with 10 removed, DETAIL (d'=1000) reaches 0.747 accuracy on Subj in about 5.2 s on one L40, versus 0.658 for IG and 0.654 for random removal.
- DETAIL scores computed on Vicuna-7b transfer to GPT-3.5, giving a reported 17.9% average accuracy gain over random curation.
- Random projection to d'=1000 gives about a 10x speedup on a 7B LLM, with wall time near 0.3 s up to that dimension before rising sharply.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DETAIL scores to reorder demonstrations when 6 demonstrations are corrupted | Vicuna-7b · Subj | accuracy: 0.588 (7.96e-03) -> 0.604 (7.39e-03) | Helped |
| Use DETAIL scores to reorder demonstrations when 6 demonstrations are corrupted | Vicuna-7b · SST-2 | accuracy: 0.487 (9.45e-03) -> 0.520 (1.01e-02) | Helped |
| Use DETAIL scores to reorder demonstrations when 6 demonstrations are corrupted | Vicuna-7b · Rotten Tomatoes | accuracy: 0.398 (1.12e-02) -> 0.425 (1.37-02) | Helped |
| Combine transformer hidden state and BERT embedding using an equal-weight average (ablation) | DETAIL curation · Subj | accuracy: 0.665 (2.36e-02) -> 0.758 (1.97e-02) | Helped |
| Combine transformer hidden state and BERT embedding using an equal-weight average (ablation) | DETAIL curation · SST-2 | accuracy: 0.475 (1.31e-02) -> 0.579 (2.34e-02) | Helped |
| Combine transformer hidden state and BERT embedding using an equal-weight average (ablation) | DETAIL curation · Rotten Tomatoes | accuracy: 0.510 (2.10e-02) -> 0.575 (2.46e-02) | Helped |
| Combine transformer hidden state and BERT embedding using an equal-weight average (ablation) | DETAIL curation · AG News | accuracy: 0.408 (2.10e-02) -> 0.425 (1.76e-02) | Helped |
| Use DETAIL scores from later transformer layers rather than earlier layers (ablation) | Vicuna-7b · Demonstration removal, perturbation, and noisy-label detection on Subj and SST-2 | task performance | Helped |
| Use the column position (immediately before the label) rather than the label position for embeddings (ablation) | DETAIL · Demonstration removal on Subj and SST-2 | accuracy | No clear effect |
| Use 10 rather than 5 LiSSA iterations (ablation) | Attribution baseline in demonstration curation · Subj | accuracy: 0.556 (1.38e-02) -> 0.597 (3.23e-02) | No clear effect |
| Use 10 rather than 5 LiSSA iterations (ablation) | Attribution baseline in demonstration curation · SST-2 | accuracy: 0.493 (1.34e-02) -> 0.499 (1.64e-02) | No clear effect |
| Use 10 rather than 5 LiSSA iterations (ablation) | Attribution baseline in demonstration curation · Rotten Tomatoes | accuracy: 0.498 (1.72e-02) -> 0.494 (1.74e-02) | No clear effect |
Surprising findings
- Combining DETAIL with BERT-score embeddings via weighted averaging improved attribution accuracy in a preliminary experiment.
- DETAIL performed inconsistently on Mamba-2.8b and poorly on the 4-way AG News curation task.
Limitations
- Requires access to the model's internal hidden states, so it does not directly apply to black-box models.
- Performance is inconsistent on Mamba (SSM), which the theory does not cover, and on the 4-way AG News curation task.
- Relies on a theoretical view of transformers as internal optimizers, with no guarantee for general LLMs; lambda must be heuristically set.
Experimental setup
Datasets: MNIST AG News SST-2 Rotten Tomatoes Subj
Benchmarks: AG News SST-2 Rotten Tomatoes Subj
Metrics: test accuracy after demonstration removal/corruption AUROC for noisy demonstration detection wall time (seconds)
Baselines: Integrated gradients LIME Attention Datamodel BERT-Score LOO
Builds on: influence functions linear self-attention internal optimizer Johnson-Lindenstrauss lemma LiSSA datamodels
Closest papers by meaning
- Capturing In-Context Learning Dynamics with Task Operators (NeurIPS 2026 · similarity 0.91)
- Technical Debt in In-Context Learning: Diminishing Efficiency in Long Context (NeurIPS 2025 · similarity 0.90)
- Can In-context Learning Really Generalize to Out-of-distribution Tasks? (ICLR 2025 · similarity 0.89)
- In-Context Learning through the Bayesian Prism (ICLR 2024 · similarity 0.89)
- Are Human-generated Demonstrations Necessary for In-context Learning? (ICLR 2024 · similarity 0.89)
- How Do Transformers Learn In-Context Beyond Simple Functions? A Case Study on Learning with Representations (ICLR 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.