Research map / Image generation / Fast diffusion sampling
Influence Functions for Scalable Data Attribution in Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Extends influence functions to diffusion models by using an (E)K-FAC approximation of a diffusion-specific generalised Gauss-Newton Hessian, outperforming TRAK-style baselines on LDS and retraining-without-top-influences evaluations.
Problem
Scalable data attribution for diffusion models: predicting how removing training data changes the loss or probability of generating a given sample, where Hessian inversion for influence functions is intractable.
Approach
Formulates a diffusion-specific GGN (model-output split, equivalent to a Fisher F_D) and approximates it with K-FAC-expand and eigenvalue-corrected K-FAC, then uses it in influence functions with loss/ELBO/trajectory-probability proxy measurements; reinterprets prior TRAK/D-TRAK methods as particular GGN (empirical Fisher) and projection choices.
Key results
- K-FAC Influence outperforms TRAK in all LDS settings (loss, ELBO, marginal log-probability on dequantised data).
- K-FAC Influence with loss measurement also outperforms the benchmark-tuned D-TRAK changes in all settings.
- K-FAC influence functions pick more influential training examples for counterfactual retraining than baselines on CIFAR-2 and CIFAR-10 (2% and 10% removed).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use K-FAC Influence instead of TRAK | Diffusion-model data attribution · LDS (loss, ELBO, and marginal log-probability measurements) | LDS | Helped |
| Use K-FAC Influence with the loss measurement instead of D-TRAK | Diffusion-model data attribution · LDS | LDS | Helped |
| Use K-FAC Influence instead of the baselines to select examples for removal | Diffusion-model data attribution · CIFAR-2 and CIFAR-10; 2% and 10% data removal | Loss reduction after retraining | Helped |
| Use eigenvalue-corrected K-FAC instead of standard K-FAC (ablation) | K-FAC Influence · Diffusion-model data attribution | Performance | Helped |
| Use quantisation rather than SVD-based gradient compression (ablation) | Diffusion-model influence estimation · Diffusion models | Gradient compression effectiveness | Helped |
| Use the loss measurement rather than the ELBO measurement when predicting changes in ELBO or marginal log-probability (ablation) | K-FAC Influence · LDS for ELBO and marginal log-probability | LDS | Helped |
| Use higher-timestep losses as proxy measurements for lower-timestep losses (ablation) | K-FAC Influence · Per-timestep loss LDS | LDS | Helped |
| Use the correct timestep loss rather than a higher-timestep loss as the proxy (ablation) | K-FAC Influence · Some per-timestep loss predictions | LDS | Hurt |
Surprising findings
- Substituting the training loss as the measurement works better for LDS on ELBO/marginal log-probability than the correct measurement.
- Higher-timestep losses act as better proxies for lower-timestep losses in influence predictions.
- For sufficiently large training sets, the diffusion model's marginal probability of generated samples is nearly constant regardless of which training examples are removed.
Limitations
- Influence predictions are inaccurate for numerical changes in marginal probability, the measurements most relevant to generative modelling.
- Influence functions overestimate how often data removal improves loss.
- Marginal probability measurements are nearly constant for large training sets, limiting attribution signal.
Experimental setup
Datasets: CIFAR-10 CIFAR-2 ArtBench
Benchmarks: CIFAR-10 CIFAR-2 ArtBench
Metrics: Linear Datamodeling Score (LDS) change in measurement under retraining without top influence
Baselines: TRAK D-TRAK CLIP cosine similarity Exact Retraining
Builds on: influence functions K-FAC EK-FAC TRAK D-TRAK
Closest papers by meaning
- Diffusion Attribution Score: Evaluating Training Data Influence in Diffusion Models (ICLR 2025 · similarity 0.92)
- Intriguing Properties of Data Attribution on Diffusion Models (ICLR 2024 · similarity 0.90)
- Training Data Attribution in Diffusion Models via Mirrored Unlearning and Noise-Consistent Skew (NeurIPS 2026 · similarity 0.90)
- A Versatile Influence Function for Data Attribution with Non-Decomposable Loss (ICML 2025 · similarity 0.89)
- Rescaled Influence Functions: Accurate Data Attribution in High Dimension (NeurIPS 2025 · similarity 0.88)
- The Emergence of Reproducibility and Consistency in Diffusion Models (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.