Research map / Learning theory and algorithms / Optimal transport
SAVA: Scalable Learning-Agnostic Data Valuation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SAVA, a batch-wise, memory-efficient variant of LAVA that computes hierarchical optimal transport data valuation scores on batches to scale data valuation to million-point datasets, and corrects an entropic-regularization gradient theorem from LAVA.
Problem
Data valuation for training-set noise and relevance using optimal transport against a clean validation set is limited by LAVA's O(N^2) memory, which prevents scaling to large web-scraped datasets.
Approach
Partition training and validation sets into batches, compute OT between batch pairs, then solve a batch-level OT (hierarchical OT) whose plan weights the per-point calibrated OT gradients, so memory scales with batch size rather than dataset size.
Key results
- SAVA scales to Clothing1M (over 1M images), where LAVA runs out of memory on large CIFAR10 subsets, and SAVA matches LAVA's detection on smaller data.
- On Clothing1M pruning, SAVA reaches 70.0 ± 0.2 test accuracy versus 67.6 ± 0.2 for training on the full noisy set, outperforming EL2N and Batch-wise LAVA.
- SAVA consistently detects noisy labels and features at 50K CIFAR10 scale, where LAVA hits OOM errors.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use SAVA to prune low-value training data | ResNet18 on Clothing1M · Clothing1M | accuracy: 67.6 \pm 0.2 -> 70.0 \pm 0.2 | Helped |
| weight within-batch OT gradients using the optimal plan across validation batches instead of uniform weighting | SAVA on Clothing1M · Clothing1M | accuracy | Helped |
| use SAVA rather than EL2N or Batch-wise LAVA for pruning | ResNet18 on Clothing1M · Clothing1M | accuracy | Helped |
| use SAVA for valuation instead of LAVA | CIFAR10 corruption detection · CIFAR10 noisy labels | detection rate | Mixed |
| use SAVA rather than TracInCP | CIFAR10 corruption detection · CIFAR10 noisy labels | detection rate | Helped |
| use SAVA rather than KNN Shapley | CIFAR10 corruption detection · CIFAR10 noisy features | detection rate | Helped |
| use SAVA rather than KNN Shapley | CIFAR10 corruption detection · CIFAR10 poison frogs attack | detection rate | Helped |
| cache label-to-label costs between validation and training batches (ablation) | SAVA · data valuation | runtime | Helped |
| cache label-to-label costs between validation and training batches (ablation) | SAVA · data valuation | performance | No clear effect |
Surprising findings
- TracInCP performs no better than random selection at detecting noisy labels.
- Batch-wise LAVA is very sensitive to batch size, which motivates SAVA's OT-weighted aggregation.
Limitations
- Hierarchical OT gives only an upper bound on OT, yielding a looser validation error bound.
- The ground cost is limited to labeled datasets; text and speech ground costs are unexplored.
- Requires a clean validation dataset.
Experimental setup
Datasets: CIFAR10 Clothing1M
Benchmarks: CIFAR10 corruption detection Clothing1M data pruning
Metrics: detection rate at N/4 prefix test accuracy after pruning
Baselines: LAVA KNN Shapley TracInCP Influence Functions Data-OOB Batch-wise LAVA EL2N Supervised prototypes
Builds on: LAVA hierarchical optimal transport Sinkhorn algorithm KNN Shapley EL2N
Closest papers by meaning
- KAIROS: Scalable Model-Agnostic Data Valuation (NeurIPS 2025 · similarity 0.87)
- Convex Dataset Valuation for Post-Training (ICML 2026 · similarity 0.87)
- DU-Shapley: A Shapley Value Proxy for Efficient Dataset Valuation (NeurIPS 2024 · similarity 0.84)
- GMValuator: Similarity-based Data Valuation for Generative Models (ICLR 2025 · similarity 0.84)
- Taming Hyperparameter Sensitivity in Data Attribution: Practical Selection Without Costly Retraining (NeurIPS 2025 · similarity 0.84)
- Repeated Random Sampling for Minimizing the Time-to-Accuracy of Learning (ICLR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.