Research map / Image generation / Autoregressive and latent generation
GMValuator: Similarity-based Data Valuation for Generative Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes GMValuator, a training-free, model-agnostic data valuation method for image generative models that scores training images by similarity (PQ recall plus DreamSim/LPIPS re-ranking) to generated samples, calibrated by MANIQA quality.
Problem
Data valuation for generative models, i.e., measuring how much each training sample contributes to a fixed set of generated images, is under-studied; existing methods are model-specific, need expensive retraining or Hessian computation, or are unreliable across metrics.
Approach
Reformulate generative data valuation as an efficient similarity-matching problem: each generated image retrieves its top-k training contributors via product-quantized CLIP embeddings, re-ranks them with perceptual distance (DreamSim or LPIPS), and distributes credit weighted by the generated image's MANIQA quality score.
Key results
- On CelebA, GMValuator (DreamSim) reaches 97.78% identical-hat attribute match at k=5 versus 92.52% for No-Rerank.
- On MNIST with beta-VAE at k=30, GMValuator (DreamSim) achieves 88.78% identical-class ratio versus 72.00% for VAE-TracIn; on CIFAR-10 it gets 77.94% vs 6.28%.
- On MNIST data valuation for GAN, GMValuator (No-Rerank) takes 2,137 s versus 14,543 s for IF4GAN.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use DreamSim for perceptual re-ranking instead of no re-ranking (ablation) | GMValuator · CIFAR-10 (β-VAE, k=30) | identical class ratio: 72.66 -> 77.94 | Helped |
| use DreamSim for perceptual re-ranking instead of no re-ranking (ablation) | GMValuator · ImageNet (k=30) | identical class ratio: 56.18 -> 77.27 | Helped |
| use LPIPS for re-ranking instead of no re-ranking (ablation) | GMValuator · CIFAR-10 (β-VAE, k=30) | identical class ratio: 72.66 -> 72.60 | No clear effect |
| use DreamSim for re-ranking instead of LPIPS (ablation) | GMValuator · CelebA Hat (k=5) | identical attribute ratio: 96.77 -> 97.78 | Helped |
| use DreamSim for re-ranking instead of LPIPS (ablation) | GMValuator · CelebA Gender (k=5) | identical attribute ratio: 97.98 -> 99.19 | Helped |
| use DreamSim for re-ranking instead of LPIPS (ablation) | GMValuator · CelebA Eyeglasses (k=5) | identical attribute ratio: 94.95 -> 96.77 | Helped |
| use DreamSim for re-ranking instead of no re-ranking (ablation) | GMValuator · MNIST, β-VAE (k=30) | identical class ratio: 86.41 -> 88.78 | Helped |
| use DreamSim for re-ranking instead of LPIPS (ablation) | GMValuator · ImageNet (k=30) | identical class ratio: 50.30 -> 77.27 | Helped |
| use DreamSim for re-ranking instead of LPIPS (ablation) | GMValuator · MNIST (VAE, attribution of one generated sample) | attribution time (s): 0.477 -> 1.709 | Hurt |
| use DreamSim for re-ranking instead of LPIPS (ablation) | GMValuator · Noise MNIST | data valuation time (s): 4,771 -> 17,086 | Hurt |
| use DreamSim for re-ranking instead of LPIPS (ablation) | GMValuator · CIFAR-10 (β-VAE, k=30) | identical class ratio: 72.60 -> 77.94 | Helped |
| use GMValuator instead of VAE-TracIn | data valuation for generative models · CIFAR-10 (β-VAE, k=30) | identical class ratio: 6.28 -> 77.94 | Helped |
Surprising findings
- Using pixel-space l2 distance in re-ranking gives comparable results to LPIPS/DreamSim on MNIST, suggesting flexibility in distance choice.
- On ImageNet, l2-distance and LPIPS re-ranking perform worse than no re-ranking, while DreamSim performs best, indicating semantic features matter at scale.
Limitations
- Evaluated only on image modality; authors note extension to other data modalities is future work.
- Validation relies on proxy ground truth (class or attribute labels) rather than true counterfactual contributions.
- Re-ranking with DreamSim is slower than other variants, creating an efficiency-accuracy trade-off.
Experimental setup
Datasets: MNIST CIFAR-10 ImageNet CelebA AFHQ FFHQ Naruto
Benchmarks: MNIST CIFAR-10 ImageNet CelebA AFHQ FFHQ Naruto
Metrics: identical class ratio identical attribute accuracy OOD noisy-sample value rank runtime (s)
Baselines: VAE-TracIn IF4GAN GMValuator (No-Rerank) GMValuator (l2-distance) GMValuator (LPIPS)
Builds on: VAE-TracIn IF4GAN KNN-Shapley DreamSim LPIPS
Closest papers by meaning
- KAIROS: Scalable Model-Agnostic Data Valuation (NeurIPS 2025 · similarity 0.85)
- Self-Consuming Generative Models with Curated Data Provably Optimize Human Preferences (NeurIPS 2024 · similarity 0.84)
- SAVA: Scalable Learning-Agnostic Data Valuation (ICLR 2025 · similarity 0.84)
- What is Your Data Worth to GPT? LLM-Scale Data Valuation with Influence Functions (NeurIPS 2025 · similarity 0.84)
- GenAI Arena: An Open Evaluation Platform for Generative Models (NeurIPS 2024 · similarity 0.83)
- On the Stability of Iterative Retraining of Generative Models on their own Data (ICLR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.