Research map / Label-efficient and robust learning / Knowledge distillation
Few-Shot Dataset Distillation via Translative Pre-Training
Songhua Liu, Xinchao Wang
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add translator pre-training on ImageNet before adaptation (ablation) | Translator adaptation · CIFAR10 IPC 10 | accuracy (%): 53.8 -> 59.2 | Improved |
| add target-dataset adaptation to pre-trained translator (ablation) | Pre-trained translator · CIFAR10 IPC 10 | accuracy (%): 54.7 -> 59.2 | Improved |
| use pre-trained translator instead of a single network without translation (ablation) | 1 Net distillation · CIFAR10 IPC 10 | accuracy (%): 50.5 -> 54.7 | Improved |
| replace translative model with bi-level algorithm (ablation) | Distillation space learning · CIFAR10 IPC 10 | accuracy (%): 59.2 -> 52.1 | Hurt |
| initialize adaptation from ImageNet pre-trained translator rather than a reconstruction auto-encoder (ablation) | Translator adaptation · CIFAR100 IPC 10 | accuracy (%): 27.0 -> 35.1 | Improved |
| few-shot method (one network + translator) versus many networks (ablation) | Ours vs Baseline · CIFAR10 IPC 10 | accuracy (%): 57.9 -> 59.2 | Improved |
| use down-sampled parameterization (Ours-DS) | Ours · CIFAR10 IPC 1 | accuracy (%): 44.0 -> 51.4 | Improved |
| few-shot translative method versus FRePo | Ours (default parameterization) · CIFAR10/CIFAR100 | accuracy (%): about 3 to 7 points behind FRePo, with 30 to 120x acceleration | Hurt |
| adaptation on small IPC generalizes to unseen IPC | Adapted translator · CIFAR10 | accuracy (%) | Improved |
Closest papers by meaning
- Large Scale Dataset Distillation with Domain Shift (ICML 2024 · similarity 0.92)
- Data Distillation Can Be Like Vodka: Distilling More Times For Better Quality (ICLR 2024 · similarity 0.92)
- Dataset Distillation for Pre-Trained Self-Supervised Vision Models (NeurIPS 2025 · similarity 0.91)
- D^4: Dataset Distillation via Disentangled Diffusion Model (CVPR 2024 · similarity 0.90)
- Boost Self-Supervised Dataset Distillation via Parameterization, Predefined Augmentation, and Approximation (ICLR 2025 · similarity 0.90)
- Unlocking Dataset Distillation with Diffusion Models (NeurIPS 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
