Research map / Learning theory and algorithms / Prediction and decision losses
Imperfect Influence, Reliable Rankings: A Theory of TRAK for Data Attribution
Han Tong, Shubhangi Ghosh, Haolin Zou, Arian Maleki
Read the paperCheck your idea against 137,217 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Provides a theoretical analysis of TRAK, a widely used data attribution method that approximates the model with a kernel machine and uses approximate leave-one-out (ALO) techniques.
Problem
Data attribution traces a model's predictions back to training examples; TRAK is empirically strong but its theoretical conditions for reliable attribution rankings were not well understood.
Approach
Theoretical analysis of TRAK as a kernel-machine approximation combined with approximate leave-one-out risk techniques, explaining when its imperfect influence estimates still yield reliable rankings of training data.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add linearization step to true leave-one-out influence (ablation) | TRAK · 3-class simulation | influence accuracy vs true | Hurt |
| add ALO approximation after linearization (ablation) | TRAK · 3-class simulation | influence error: 1.49 -> 1.49 | No clear effect |
| reduce random projection dimension from 0.75d to 0.25d (ablation) | TRAK · 3-class simulation | Corr. 5%/Uncorr. 95% separation: 1.42 -> 0.89 | Hurt |
| reduce random projection dimension to 0.1d (ablation) | TRAK · 3-class simulation | Corr. 5%/Uncorr. 95% separation: 1.42 -> 0.42 | Hurt |
| apply linearization and ALO (no projection) (ablation) | TRAK · 3-class simulation | separation ratio: 1.66 -> 1.49 | No clear effect |
| reduce projection dimension (ablation) | TRAK · 3-class simulation | correlation of TRAK influence with true influence | Hurt |
Experimental setup
Builds on: TRAK
Closest papers by meaning
- Taming Hyperparameter Sensitivity in Data Attribution: Practical Selection Without Costly Retraining (NeurIPS 2025 · similarity 0.87)
- $\texttt{dattri}$: A Library for Efficient Data Attribution (NeurIPS 2024 · similarity 0.86)
- Final-Model-Only Data Attribution with a Unifying View of Gradient-Based Methods (NeurIPS 2025 · similarity 0.85)
- quanda: An Interpretability Toolkit for Training Data Attribution Evaluation (NeurIPS 2026 · similarity 0.84)
- Diffusion Models for Attribution (AAAI 2025 · similarity 0.84)
- Faithful and Efficient Explanations for Neural Networks via Neural Tangent Kernel Surrogate Models (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
