Research map / Label-efficient and robust learning / Recognition and long-tail
Ameliorate Spurious Correlations in Dataset Condensation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Studies how dataset bias (color, background, corruption) propagates through dataset condensation and proposes a KDE-based sample reweighting, using a supervised contrastive embedding, to suppress bias amplification in DM and DSA.
Problem
Dataset condensation can amplify spurious correlations (color and background bias) present in the original dataset, degrading models trained on the condensed set when tested on unbiased data.
Approach
Reweight real-sample embeddings in the distribution-matching or gradient-matching objective by the inverse of a kernel density estimate computed in a supervised contrastive embedding space, so bias-aligned clusters are down-weighted and bias-conflicting samples get more influence.
Key results
- On CMNIST with 5% bias-conflict and IPC 50, DM+Ours reaches 91.5% vs 23.8% for vanilla DM and 53.7% for SelecMix applied to the DM synthetic set.
- DSA+Ours on CMNIST with 1% bias-conflict and IPC 50 improves accuracy from 14.5% to 81.4%.
- On BFFHQ, the method improves accuracy by up to 6.2% despite only 0.5% bias-conflicting samples.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add KDE-based sample reweighting to DM | DM · CMNIST, 5% bias-conflict, IPC 50 | test accuracy: 23.8% to 91.5% | Helped |
| Add KDE-based sample reweighting to DSA | DSA · CMNIST, 5% bias-conflict, IPC 50 | test accuracy: 68.5% to 94.0% | Helped |
| Add KDE-based sample reweighting to DM | DM · BG FMNIST, 5% bias-conflict, IPC 50 | test accuracy: 44.6 to 65.0 | Helped |
| Add KDE-based sample reweighting to DM | DM · Corrupted CIFAR-10, 5% bias-conflict, IPC 50 | test accuracy: 38.7±0.4 -> 40.2±0.4 | Helped |
| Add KDE-based sample reweighting to DSA | DSA · BFFHQ, IPC 50 | test accuracy: 58.7±1.4 -> 61.5±0.7 | Helped |
| Apply DFA debiasing during training on DM synthetic data (ablation) | DM synthetic data · CMNIST, 5% bias-conflict, IPC 50 | test accuracy: 23.8±1.3 -> 31.2±1.3 | Helped |
| Apply SelecMix debiasing during training on DM synthetic data (ablation) | DM synthetic data · CMNIST, 5% bias-conflict, IPC 50 | test accuracy: 23.8±1.3 -> 53.7±1.5 | Helped |
| Apply DFA debiasing during training on DM synthetic data (ablation) | DM synthetic data · BG FMNIST, 5% bias-conflict, IPC 50 | test accuracy: 44.6±0.9 -> 17.6±1.9 | Hurt |
| Apply SelecMix debiasing during training on DM synthetic data (ablation) | DM synthetic data · BG FMNIST, 5% bias-conflict, IPC 50 | test accuracy: 44.6±0.9 -> 58.7±0.9 | Helped |
| Apply DFA to DM embeddings and match intrinsic features only (ablation) | DM · CMNIST, 5% bias-conflict, IPC 50 | test accuracy: 23.8±1.3 -> 25.4±0.4 | Helped |
| Use SelecMix-debiased expert training trajectories (ablation) | MTT · CMNIST, 5% bias-conflict, IPC 50 | test accuracy: 33.9±1.2 -> 52.1±0.3 | Helped |
| Debias the dataset with SelecMix before applying DM (ablation) | DM · CMNIST, IPC 10 | test accuracy: 18.6±0.2 -> 44.6±1.0 | Helped |
Surprising findings
- Color and background bias are amplified by dataset condensation, while corruption bias is suppressed, with condensed sets performing relatively better than full-data training on corrupted CIFAR-10.
- Applying state-of-the-art de-biasing methods (SelecMix, DFA) to the biased synthetic set often fails to recover performance, and DFA degrades BG FMNIST accuracy substantially.
Limitations
- Not directly applicable to trajectory-matching methods such as MTT, which do not use real images during distillation.
- Still falls short of unbiased training on full data (>85% for CMNIST) and gains are small on corrupted CIFAR-10 and BFFHQ.
- Requires a separately trained supervised contrastive model for the embedding used in KDE.
Experimental setup
Datasets: CMNIST Corrupted CIFAR-10 BFFHQ BG FMNIST MNIST Fashion-MNIST MiniPlaces
Benchmarks: CMNIST BG FMNIST Corrupted CIFAR-10 BFFHQ
Metrics: test accuracy on unbiased test set
Baselines: DM DSA MTT DREAM IDM SelecMix DFA LfF
Builds on: Distribution Matching (DM) DSA SelecMix DFA supervised contrastive learning
Closest papers by meaning
- Elucidating the Design Space of Dataset Condensation (NeurIPS 2024 · similarity 0.89)
- A Simple Remedy for Dataset Bias via Self-Influence: A Mislabeled Sample Perspective (NeurIPS 2024 · similarity 0.87)
- The Impact of Coreset Selection on Spurious Correlations and Group Robustness (NeurIPS 2025 · similarity 0.86)
- Diversity-Driven Synthesis: Enhancing Dataset Distillation through Directed Weight Adjustment (NeurIPS 2024 · similarity 0.85)
- Training Unbiased Diffusion Models From Biased Dataset (ICLR 2024 · similarity 0.85)
- Mitigating Spurious Correlations via Disagreement Probability (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.