Research map / Label-efficient and robust learning / Recognition and long-tail
Feature Distribution Matching by Optimal Transport for Effective and Robust Coreset Selection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FDMat, a coreset selection method that matches the coreset's feature distribution to the full dataset by solving an entropy-regularized optimal transport (Sinkhorn 1-Wasserstein) problem in feature space, improving robustness to noisy labels and class imbalance.
Problem
Coreset selection suffers from distribution bias (skewed feature distribution between the coreset and the full dataset), which degrades generalization under extreme pruning and in the presence of noisy or class-imbalanced data.
Approach
Relates the MMD between coreset and full dataset to the 1-Wasserstein distance via Kantorovich-Rubinstein duality with a 1-Lipschitz feature function, and selects samples whose transport cost to per-class feature centroids (after Tukey ladder-of-powers transform) is smallest, solved with Sinkhorn.
Key results
- On CIFAR-10 at 10% pruning with ResNet-18, FDMat reaches 80.29% accuracy, 3.7 points above the second-best method.
- On CIFAR-100 at 10% pruning, FDMat reaches 38.25%, 4.3 points above the second-best method (LCMat-S, 36.66%).
- FDMat's selection time on CIFAR-10 (31.96 s at 10%) is far below GradMatch (1395.73 s) and LCMat-S (803.45 s).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Select coresets by matching feature distributions with optimal transport (FDMat) instead of using LCMat-S | ResNet-18 coreset selection · CIFAR-10, 10% fraction | accuracy: 77.41±2.0 -> 80.29±2.1 | Helped |
| Select coresets by matching feature distributions with optimal transport (FDMat) instead of using LCMat-S | ResNet-18 coreset selection · CIFAR-100, 10% fraction | accuracy: 36.66±1.0 -> 38.25±1.2 | Helped |
| Select coresets by matching feature distributions with optimal transport (FDMat) instead of using LCMat-S | ResNet-18 coreset selection · Tiny-ImageNet, 10% fraction | accuracy: 13.04±0.3 -> 13.88±0.6 | Helped |
| Select coresets by matching feature distributions with optimal transport (FDMat) instead of using AdaCore | ResNet-18 coreset selection · Path-MNIST, 10% fraction | accuracy: 83.10±0.2 -> 83.96±0.2 | Helped |
| Use FDMat coreset selection rather than gradient-based selection | CIFAR-10 with mislabeled samples · CIFAR-10, mislabeled samples, 10% fraction | accuracy | Helped |
| Use FDMat coreset selection rather than other coreset methods | CIFAR-10 with class imbalance · CIFAR-10, class-imbalanced samples | accuracy | Helped |
| Use FDMat-selected samples instead of other coreset methods' samples | Cross-architecture training on CIFAR-100 · CIFAR-100, 10% and 20% fractions | accuracy | Helped |
Surprising findings
- Random selection is competitive with most existing coreset methods under extreme pruning (10%-30%) on CIFAR-10/100.
- Under 50% mislabeled data, FDMat at 10% fraction is reported to be nearly 50% better than the second-best method.
Limitations
- Relies on a Gaussian-like assumption for the feature distribution, approximated by class centroids after a Tukey transform with a hand-set beta=0.5.
- Hyperparameter lambda (entropy regularization) must be tuned; best values differ across datasets (5 on CIFAR-10, 10 on Tiny-ImageNet).
Experimental setup
Datasets: CIFAR-10 CIFAR-100 Tiny-ImageNet Path-MNIST MNIST
Benchmarks: CIFAR-10 CIFAR-100 Tiny-ImageNet Path-MNIST
Metrics: test accuracy selection time (seconds)
Baselines: Random C-Div Herding K-Center Least Confidence Entropy Margin Craig GradMatch Glister AdaCore LCMat-S
Builds on: Craig GradMatch AdaCore LCMat Sinkhorn distances
Closest papers by meaning
- Refined Coreset Selection: Towards Minimal Coreset Size under Model Performance Constraints (ICML 2024 · similarity 0.88)
- Contributing Dimension Structure of Deep Feature for Coreset Selection (AAAI 2024 · similarity 0.87)
- $\mathbb{D}^2$ Pruning: Message Passing for Balancing Diversity & Difficulty in Data Pruning (ICLR 2024 · similarity 0.86)
- Stable Coresets via Posterior Sampling: Aligning Induced and Full Loss Landscapes (NeurIPS 2025 · similarity 0.86)
- The Impact of Coreset Selection on Spurious Correlations and Group Robustness (NeurIPS 2025 · similarity 0.85)
- Mind the Boundary: Coreset Selection via Reconstructing the Decision Boundary (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.