Research map / Large language models / AI and society
$\texttt{dattri}$: A Library for Efficient Data Attribution
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces dattri, an open-source PyTorch library offering a unified API, modular low-level utilities, and a benchmark suite with pre-trained checkpoints and ground-truth annotations for efficient data attribution methods.
Problem
Data attribution research lacks a comprehensive library that supports development, benchmarking, and deployment of efficient data attribution methods, with no unified API or standardized benchmarks including generative settings.
Approach
A unified attributor/task API requiring minimal code changes in PyTorch pipelines, modular IHVP/HVP/random projection utilities, and a benchmark suite spanning image classification and generative (music, text) settings with pre-trained LOO/LDS ground-truth checkpoints.
Key results
- On LR/MLP on MNIST-10, IF variants (except Arnoldi) perform best among methods, and TRAK-10/50 are comparable to IF on LR.
- On MLP, none of the methods work well in LOO and TRAK-50 achieves the best LDS.
- On larger models (ResNet-9 on CIFAR-2, Music Transformer, NanoGPT), only TRAK achieves non-trivial LDS with ensembling; IF variants are infeasible due to computational cost.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use IF methods rather than other attribution methods | Data attribution methods · MNIST-10 with logistic regression | LOO and LDS | Helped |
| Ensemble 10 or 50 models with TRAK rather than use TRAK-1 | TRAK · MNIST-10 with logistic regression | LOO and LDS | Helped |
| Use TRAK-50 | Data attribution methods · MNIST-10 with MLP | LDS | Helped |
| Use data attribution methods on MLP instead of logistic regression | Data attribution methods · MNIST-10 | LOO | Hurt |
| Use ensembling with TRAK | TRAK · ResNet-9 on CIFAR-2, Music Transformer on MAESTRO, and NanoGPT on Shakespeare | LDS | Helped |
| Use TRAK rather than other attribution methods | Data attribution methods · ResNet-9 on CIFAR-10 | AUC | Helped |
| Normalize gradients in Grad-Cos | Grad-Dot · Noisy label detection | AUC | Hurt |
| Increase the number of TRAK ensemble models | TRAK · MNIST-10 + MLP and CIFAR-2 + ResNet-9 | runtime: 1.83s -> 13.71s -> 67.74s; 36.98s -> 316.22s -> 1549.24s | Hurt |
Surprising findings
- Grad-Cos is theoretically no better than random for noisy label detection since self-influence normalizes to 1.
- On CIFAR-10 noisy label detection, only TRAK achieves non-trivial AUC, with a significant drop for all methods.
Limitations
- Missing many efficient data attribution methods and benchmark datasets, as acknowledged by the authors.
- Game-theoretic methods such as Data Shapley and Data Banzhaf are excluded.
- AUC metric is tied to noisy label detection and only applicable to classification; LOO and LDS require many retrained models.
Experimental setup
Datasets: MNIST-10 CIFAR-10 CIFAR-2 MAESTRO Tiny Shakespeare
Benchmarks: MNIST-10 LOO/LDS/AUC CIFAR-2 LDS CIFAR-10 AUC MAESTRO LDS Shakespeare LDS
Metrics: LOO correlation linear datamodeling score AUC for noisy label detection runtime peak memory
Baselines: OpenDataVal pyDVL Influenciae Captum
Builds on: influence functions TracIn TRAK Representer Point Selection torch.func
Closest papers by meaning
- quanda: An Interpretability Toolkit for Training Data Attribution Evaluation (NeurIPS 2026 · similarity 0.89)
- Taming Hyperparameter Sensitivity in Data Attribution: Practical Selection Without Costly Retraining (NeurIPS 2025 · similarity 0.88)
- Final-Model-Only Data Attribution with a Unifying View of Gradient-Based Methods (NeurIPS 2025 · similarity 0.86)
- DATE-LM: Benchmarking Data Attribution Evaluation for Large Language Models (NeurIPS 2025 · similarity 0.85)
- Fast Data Attribution for Text-to-Image Models (NeurIPS 2025 · similarity 0.85)
- Diffusion Models for Attribution (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.