Research map / Image generation / Fast diffusion sampling
Diffusion Attribution Score: Evaluating Training Data Influence in Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Diffusion Attribution Score (DAS), a KL-based training-data attribution method for diffusion models that directly compares noise predictors of full and leave-one-out models via a Newton-step/Sherman-Morrison approximation, with projection, normalization and compression for scalability.
Problem
Data attribution for diffusion models: existing TRAK/D-TRAK-style scores use diffusion loss, which indirectly compares predicted distributions against the data distribution and fails to capture changes in generated content.
Approach
Replace the diffusion-loss output function with the noise predictor output and measure the KL divergence between the predicted distributions with and without a training sample, approximating parameter change by a single Newton step with a Sherman-Morrison closed form, plus projection/normalization/model-compression to scale.
Key results
- On CIFAR-2 validation (100 timesteps), DAS reaches 43.08% LDS vs 33.74% for D-TRAK and 23.59% for TRAK.
- On ArtBench-2 generation (100 timesteps), DAS reaches 32.31% LDS vs 26.53% for D-TRAK.
- On CelebA generation (100 timesteps), DAS reaches 30.68% LDS vs 21.47% for D-TRAK.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace D-TRAK's loss-based attribution with DAS's direct noise-predictor comparison | Diffusion data attribution · CIFAR-2 validation, 100 timesteps | LDS (%): +9.33% | Helped |
| Replace D-TRAK's loss-based attribution with DAS's direct noise-predictor comparison | Diffusion data attribution · CIFAR-2 generation, 100 timesteps | LDS (%): +5.01% | Helped |
| Replace D-TRAK's loss-based attribution with DAS's direct noise-predictor comparison | Diffusion data attribution · ArtBench-2 validation, 100 timesteps | LDS (%): +8.39% | Helped |
| Replace D-TRAK's loss-based attribution with DAS's direct noise-predictor comparison | Diffusion data attribution · ArtBench-2 generation, 100 timesteps | LDS (%): +5.78% | Helped |
| Replace D-TRAK's loss-based attribution with DAS's direct noise-predictor comparison | Diffusion data attribution · CIFAR-10 validation, 100 timesteps | LDS (%): +12.67% | Helped |
| Replace D-TRAK's loss-based attribution with DAS's direct noise-predictor comparison | Diffusion data attribution · CIFAR-10 generation, 100 timesteps | LDS (%): +7.58% | Helped |
| Replace D-TRAK's loss-based attribution with DAS's direct noise-predictor comparison | Diffusion data attribution · ArtBench-5 validation, 100 timesteps | LDS (%): +10.21% | Helped |
| Replace D-TRAK's loss-based attribution with DAS's direct noise-predictor comparison | Diffusion data attribution · ArtBench-5 generation, 100 timesteps | LDS (%): +8.92% | Helped |
| Use noise-predictor differences instead of loss differences to assess generation changes | CIFAR-2 DDPM · CIFAR-2 generated image pairs | Pearson correlation with image L2 distance: 0.257 -> 0.485 | Helped |
Surprising findings
- Diffusion loss differences correlate weakly with generated-image changes (Pearson 0.257) whereas noise predictor differences correlate more strongly (0.485).
- Modest improvements on CIFAR-10 and CelebA are attributed to using only one random seed per subset in LDS evaluation, which may make LDS inaccurate.
Limitations
- Modest gains on CIFAR-10 and CelebA, possibly due to single-seed LDS evaluation.
- Performance on generation set lower than validation set; the paper suggests generated image quality may matter but leaves it uninvestigated.
- Full DAS is computationally expensive, requiring speed-up approximations that trade off accuracy.
Experimental setup
Datasets: CIFAR-10 CIFAR-2 CelebA ArtBench-2 ArtBench-5
Benchmarks: CIFAR-2 CIFAR-10 CelebA ArtBench-2 ArtBench-5
Metrics: LDS (Spearman correlation, %) L2 distance CLIP cosine similarity
Baselines: Raw pixel similarity CLIP similarity Gradient similarity TracInCP GAS Journey TRAK Relative IF Renormalized IF TRAK D-TRAK
Builds on: TRAK D-TRAK Journey TRAK Linear Datamodeling Score DDPM
Closest papers by meaning
- Intriguing Properties of Data Attribution on Diffusion Models (ICLR 2024 · similarity 0.94)
- Influence Functions for Scalable Data Attribution in Diffusion Models (ICLR 2025 · similarity 0.92)
- Training Data Attribution in Diffusion Models via Mirrored Unlearning and Noise-Consistent Skew (NeurIPS 2026 · similarity 0.91)
- An Efficient Framework for Crediting Data Contributors of Diffusion Models (ICLR 2025 · similarity 0.90)
- The Emergence of Reproducibility and Consistency in Diffusion Models (ICML 2024 · similarity 0.89)
- One-Step Diffusion Distillation through Score Implicit Matching (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.