Research map / Large language models / Instruction and fine-tuning
Helpful or Harmful Data? Fine-tuning-free Shapley Attribution for Explaining Language Model Predictions
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a sign-based robustness notion for instance attribution and FreeShap, a fine-tuning-free approximation of the Shapley value that uses empirical NTK kernel regression to score training examples for BERT/RoBERTa and Llama2.
Problem
Instance attribution scores for fine-tuned language models are dataset-dependent, so the sign of a training example's score (helpful vs harmful) can flip when the rest of the dataset is resampled; LOO-based methods are non-robust and exact Shapley values require exponentially many fine-tuning runs.
Approach
Define beta-robustness of the sign of instance scores, show theoretically and empirically that the Shapley value is more robust than leave-one-out, and make Shapley computation cheap by replacing each subset's fine-tuning with eNTK kernel regression using submatrices of one precomputed kernel matrix.
Key results
- FreeShap correlates with MC-Shapley at Pearson 0.70 and Spearman 0.60 on SST-2 (500 examples), versus 0.44 and 0.48 for G-Shapley.
- Shapley-based scores are sign-inconsistent less often than LOO; e.g., at |D_N|=500 on SST-2, non-robust occurrences are 30% for Shapley vs 86% for LOO.
- On MRPC with |D_N|=3000, non-robust occurrence is 4% for Shapley vs 98% for LOO.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace G-Shapley with FreeShap's eNTK-based Shapley approximation | BERT instance attribution · SST-2, MR, MRPC, and RTE (500 examples) | correlation with MC-Shapley: SST-2 Pearson 0.44$\pm$0.08 -> 0.70$\pm$0.08; MR Pearson 0.32$\pm$0.22 -> 0.57$\pm$0.15; MRPC Pearson 0.68$\pm$0.08 -> 0 | Helped |
| use Shapley value instead of LOO | BERT instance attribution · MR, MRPC, and RTE (500 training examples) | non-robust occurrences: MR 98% -> 18%; MRPC 96% -> 28%; RTE 86% -> 56% | Helped |
| use Shapley value instead of LOO | BERT instance attribution · SST-2 (500 training examples) | non-robust occurrences: 86% -> 30% | Helped |
| select high-score examples using FreeShap rather than random selection | BERT · MR data selection, 2% subset | performance difference from 0-shot: 0.1970 -> 0.1951 | No clear effect |
| select high-score examples using FreeShap rather than the strongest baseline | BERT · MR data selection, 4% subset | performance difference from 0-shot: 0.1904 -> 0.2111 | Helped |
| select high-score examples using FreeShap rather than the strongest baseline | BERT · MR data selection, 6% subset | performance difference from 0-shot: 0.2073 -> 0.2148 | Helped |
| select high-score examples using FreeShap rather than the strongest baseline | BERT · MR data selection, 8% subset | performance difference from 0-shot: 0.2017 -> 0.2167 | Helped |
| select high-score examples using FreeShap rather than the strongest baseline | BERT · MR data selection, 10% subset | performance difference from 0-shot: 0.1970 -> 0.2223 | Helped |
| use one K_0 kernel instead of class-specific K_c kernels (ablation) | FreeShap · SST-2 | Pearson correlation with MC-Shapley: $0.69$ -> $0.71$ | No clear effect |
| use blockwise inversion instead of exact matrix inversion (ablation) | FreeShap · SST-2 (5k training points) | correlation with exact inverse: $0.9984$ Pearson; $0.9963$ Spearman | No clear effect |
| remove the lowest-scoring 10% of examples using FreeShap scores | BERT instance attribution · SST-2, MR, MRPC, and RTE | test performance after data removal | Mixed |
| inspect mislabeled examples in ascending FreeShap score order instead of baseline attribution order | BERT instance attribution · wrong-label detection across 8 experiments | effectiveness of mislabeled-data identification: 7 out of 8 experiments | Helped |
Surprising findings
- Exact LOO scores are sign-inconsistent for most examples in many settings (up to 100% non-robust occurrence), indicating LOO signs are unreliable under resampling.
Limitations
- Limited to NLP classification tasks; extension to text generation is left for future work.
- Requires white-box access to the model.
- Robustness comparison relies on the assumptions in the theorem (variance and diminishing influence) and empirical estimates from 5 resampled datasets.
Experimental setup
Datasets: SST-2 MR MRPC RTE
Benchmarks: SST-2 MR MRPC RTE
Metrics: Pearson correlation Spearman correlation non-robust occurrence percentage test accuracy under data removal data selection accuracy gain detected poisoned data percentage
Baselines: G-Shapley Influence function TracIn Representer point LOO MC-Shapley
Builds on: neural tangent kernel TMC-Shapley influence function TracIn representer point
Closest papers by meaning
- InstaSHAP: Interpretable Additive Models Explain Shapley Values Instantly (ICLR 2025 · similarity 0.87)
- Data Shapley in One Training Run (ICLR 2025 · similarity 0.86)
- Fine-tuning can Help Detect Pretraining Data from Large Language Models (ICLR 2025 · similarity 0.85)
- WAGLE: Strategic Weight Attribution for Effective and Modular Unlearning in Large Language Models (NeurIPS 2024 · similarity 0.85)
- Scalable Influence and Fact Tracing for Large Language Model Pretraining (ICLR 2025 · similarity 0.85)
- Localizing Input Uncertainty Quantification for Large Language Models via Shapley Values (NeurIPS 2026 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.