Research map / Image generation / Text-to-image diffusion
Training Data Provenance Verification: Did Your Model Use Synthetic Data from My Generative Model for Training?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TrainProVe, a black-box method that verifies whether a suspect's image classifier was trained on synthetic data from the defender's text-to-image model, using a shadow model and a one-sided Grubbs' hypothesis test on batch accuracies.
Problem
Detecting whether a suspicious task-specific model (e.g., an image classifier) was trained on synthetic data generated by the defender's open-source text-to-image model, with only black-box label access.
Approach
Grounded in a generalization error bound (domain adaptation H-divergence), a shadow model is trained on the defender's own generated data; if the suspect's batch accuracies on a validation set generated by the defender's model are statistically similar to the shadow model's, the suspect is deemed to have used that data source.
Key results
- TrainProVe achieves average accuracy 0.997, F1 0.992, AUROC 0.998 on CIFAR10, versus 0.768 accuracy for Han et al.'s work.
- On CIFAR100 TrainProVe reaches average accuracy 0.992, while Han et al. reach 0.730.
- On ImageNet-100 TrainProVe reaches average accuracy 0.786 versus 0.671 for Han et al.; TrainProVe's total runtime is 35.1h vs 146.9h for Han et al.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TrainProVe instead of Han et al.'s Work | training-data provenance verification · CIFAR10 | accuracy: 0.768 -> 0.997 | Helped |
| Use TrainProVe instead of Han et al.'s Work | training-data provenance verification · CIFAR100 | accuracy: 0.730 -> 0.992 | Helped |
| Use TrainProVe instead of Han et al.'s Work | training-data provenance verification · ImageNet-100 | accuracy: 0.671 -> 0.786 | Helped |
| Replace ResNet18 shadow model with ConvNeXt-B (ablation) | TrainProVe · CIFAR10 | accuracy: 0.997 -> 0.995 | No clear effect |
| Replace ResNet18 shadow model with Swin-B (ablation) | TrainProVe · CIFAR10 | accuracy: 0.997 -> 0.997 | No clear effect |
| Replace ResNet18 shadow model with ConvNeXt-B (ablation) | TrainProVe · ImageNet-100 | accuracy: 0.786 -> 0.771 | No clear effect |
| Replace ResNet18 shadow model with Swin-B (ablation) | TrainProVe · ImageNet-100 | accuracy: 0.786 -> 0.787 | No clear effect |
| Swap shadow-dataset and validation-dataset text prompts (ablation) | TrainProVe · CIFAR10 | accuracy: 0.988 -> 0.988 | No clear effect |
| Fine-tune Stable Diffusion v1.4 on real data before generating suspect training data (ablation) | TrainProVe · ImageNet-100 | accuracy: 1.00 -> 0.97 | Hurt |
| Fine-tune the suspect model on real data (ablation) | TrainProVe · ImageNet-100 | accuracy: 1.00 -> 0.89 | Hurt |
Surprising findings
- TrainProVe-Ent and TrainProVe-Sim perform near or below random on CIFAR100 and ImageNet-100, showing that comparing prediction statistics without true labels is insufficient.
Limitations
- ImageNet-100 accuracy is substantially lower (0.786) and F1 is only 0.495, indicating weaker performance on higher-resolution data.
- Evaluated only on image classification with a single defender model per experiment; other task types are not covered.
- Fine-tuning the suspect model with real data reduces accuracy notably (0.89 vs 1.00 in one setting).
Experimental setup
Datasets: CIFAR10 CIFAR100 ImageNet-100 CIFAR10-Syn CIFAR100-Syn ImageNet-100-Syn
Benchmarks: CIFAR10-Syn CIFAR100-Syn ImageNet-100-Syn
Metrics: accuracy F1 score AUROC
Baselines: Random classification Han et al.'s work TrainProVe-Ent TrainProVe-Sim
Builds on: generalization error bound (domain adaptation) Grubbs' test Han et al. attributor
Closest papers by meaning
- The Unmet Promise of Synthetic Training Images: Using Retrieved Real Images Performs Better (NeurIPS 2024 · similarity 0.86)
- Real-Fake: Effective Training Data Synthesis Through Distribution Matching (ICLR 2024 · similarity 0.86)
- On the Stability of Iterative Retraining of Generative Models on their own Data (ICLR 2024 · similarity 0.86)
- Improved Techniques for Training Consistency Models (ICLR 2024 · similarity 0.84)
- Scaling Laws of Synthetic Images for Model Training ... for Now (CVPR 2024 · similarity 0.84)
- Dealing with Synthetic Data Contamination in Online Continual Learning (NeurIPS 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.