Research map / Label-efficient and robust learning / Recognition and long-tail
Contributing Dimension Structure of Deep Feature for Coreset Selection
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a Contributing Dimension Structure (CDS) metric and CDS diversity constraint that, applied to five coreset selection methods, improves them on CIFAR-10/100 and TinyImageNet with ResNet-18.
Problem
Existing feature similarity metrics (e.g., L2 distance) aggregate per-dimension differences and therefore fail to capture sample diversity in coreset selection, leading to subsets with few distinct contributing-dimension patterns and reduced model accuracy.
Approach
Binarize per-dimension deviations of PCA-reduced deep features from the class centroid (threshold beta) to form a Contributing Dimension Structure code per sample, then enforce diversity over these codes via hard (clustering-based budget allocation) or soft (penalty in the objective) constraints on existing selection methods.
Key results
- On CIFAR-10 at 1% sampling, GC+Ours reaches 43.1% vs 42.0% for GC, and at 0.5% reaches 36.4% vs 34.0%.
- LC+Ours outperforms LC by an average of 15.0% accuracy when sampling 0.1% to 20% of CIFAR-10, per the authors.
- Soft CDS improves GC by an average of 4.0% on CIFAR-10 class-imbalanced sampling from 0.1% to 20%.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add Soft CDS constraint to GC | GC · CIFAR-10 test, 0.5% sampling, class-balanced | accuracy: 34.0 -> 36.4; 2.4 ↑ | Helped |
| Add CDS constraint to M-DS | M-DS · CIFAR-10 test, 0.5% sampling, class-balanced | accuracy: 31.8 -> 33.0; 1.2 ↑ | Helped |
| Add feature similarity and CDS constraint using L2 norm (ablation) | GC · CIFAR-10 test, 0.5% sampling | accuracy: 34.0 -> 32.8; 1.2% | Hurt |
| Add dimension reduction to the L2-based CDS-constraint variant (ablation) | GC · CIFAR-10 test, 0.5% sampling | accuracy: 32.8 -> 34.3; 1.5% | Helped |
| Add partition to the dimension-reduced variant (ablation) | GC · CIFAR-10 test, 0.5% sampling | accuracy: 34.3 -> 33.5 | Hurt |
| Replace L2 norm with the CDS metric to compute CDS relationships (ablation) | GC with CDS constraint · CIFAR-10 test, 0.5% sampling | accuracy: 33.5 -> 36.4 | Helped |
| Use Hard CDS constraint | CRAIG and GC · CIFAR-10, class-balanced sampling | accuracy | No clear effect |
| Use Soft CDS constraint designed for the objective function | CRAIG and GC · CIFAR-10, class-balanced sampling | accuracy | Helped |
| Sample more data with different CDS rather than the same CDS | CIFAR-10 coreset selection · CIFAR-10, sampling rates 0.1%-10% | accuracy | Helped |
Surprising findings
- Sampling more data with the same CDS performs worse than random sampling at low sampling rates, while sampling more different-CDS data outperforms random.
- Hard CDS constraint fails to improve CRAIG and GC on CIFAR-10, requiring the soft constraint design for those methods.
Limitations
- The paper notes that the current selection pipeline does not account for the time cost of selecting data.
- Hard CDS does not improve all methods; soft designs must be hand-crafted per objective.
- The threshold beta and reduced dimension k are hyperparameters tuned per method and dataset.
Experimental setup
Datasets: CIFAR-10 CIFAR-100 TinyImageNet
Benchmarks: CIFAR-10 CIFAR-100 TinyImageNet
Metrics: classification accuracy
Baselines: Random K-Center Greedy Forgetting Least Confidence CRAIG Cal Glister Graph Cut Moderate-DS
Builds on: CRAIG Graph Cut Moderate-DS k-center greedy DeepCore
Closest papers by meaning
- $\mathbb{D}^2$ Pruning: Message Passing for Balancing Diversity & Difficulty in Data Pruning (ICLR 2024 · similarity 0.87)
- The Impact of Coreset Selection on Spurious Correlations and Group Robustness (NeurIPS 2025 · similarity 0.87)
- Balancing Feature Similarity and Label Variability for Optimal Size-Aware One-shot Subset Selection (ICML 2024 · similarity 0.87)
- Stable Coresets via Posterior Sampling: Aligning Induced and Full Loss Landscapes (NeurIPS 2025 · similarity 0.87)
- Feature Distribution Matching by Optimal Transport for Effective and Robust Coreset Selection (AAAI 2024 · similarity 0.87)
- Refined Coreset Selection: Towards Minimal Coreset Size under Model Performance Constraints (ICML 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.