Research map / Label-efficient and robust learning / Knowledge distillation
A Comprehensive Survey of Dataset Distillation
Shiye Lei, Dacheng Tao
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use dataset distillation instead of random selection | MTT · CIFAR-10 (IPC 10) | accuracy: 26.0 -> 65.3 | Improved |
| use distillation instead of herding coreset selection | KFS · CIFAR-10 (IPC 10) | accuracy: 31.6 -> 72.0 | Improved |
| add code-decoder factorization to gradient matching (DC to IDC) | DC · CIFAR-10 (IPC 10) | accuracy: 44.9 -> 67.5 | Improved |
| add factorization on the distribution matching line (DM to KFS) | DM · CIFAR-10 (IPC 10) | accuracy: 48.9 -> 72.0 | Improved |
| add factorization (DM to IDM) | DM · CIFAR-100 (IPC 10) | accuracy: 29.7 -> 45.1 | Improved |
| transfer distilled data from ConvNet to ResNet-152 | MTT · CIFAR-10 (IPC 10) | accuracy: 65.3 -> 15.9 | Hurt |
| transfer distilled data from ConvNet to ViT | DM · CIFAR-10 (IPC 10) | accuracy: 48.9 -> 34.4 | Hurt |
| transfer distilled data from ConvNet to DenseNet-121 | IDC (factorized) · CIFAR-10 (IPC 10) | accuracy: 67.5 -> 61.6 | Hurt |
| transfer distilled data from ConvNet to DenseNet-121 | KFS (factorized) · CIFAR-10 (IPC 10) | accuracy: 72.0 -> 71.4 | No clear effect |
| transfer distilled data from ConvNet to ResNet-18 | KIP · CIFAR-10 (IPC 1) | accuracy: 49.9 -> 27.6 | Hurt |
| use KRR matching instead of gradient matching (DC to RCIG) | DC · CIFAR-100 (IPC 10) | accuracy: 26.6 -> 44.1 | Improved |
| use trajectory matching instead of distribution matching (DM to MTT) | DM · Tiny ImageNet (IPC 10) | accuracy: 12.9 -> 23.2 | Improved |
Closest papers by meaning
- Large Scale Dataset Distillation with Domain Shift (ICML 2024 · similarity 0.92)
- Boost Self-Supervised Dataset Distillation via Parameterization, Predefined Augmentation, and Approximation (ICLR 2025 · similarity 0.91)
- Rethinking Dataset Distillation for Classification: Do Distilled Sets Outperform Coresets? (NeurIPS 2026 · similarity 0.91)
- Data Distillation Can Be Like Vodka: Distilling More Times For Better Quality (ICLR 2024 · similarity 0.90)
- Going Beyond Feature Similarity: Effective Dataset distillation based on Class-aware Conditional Mutual Information (ICLR 2025 · similarity 0.90)
- Provable and Efficient Dataset Distillation for Kernel Ridge Regression (NeurIPS 2024 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
