Research map / Label-efficient and robust learning / Recognition and long-tail
Optimizing Distributional Geometry Alignment with Optimal Transport for Generative Dataset Distillation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Reformulates generative dataset distillation as optimal transport distance minimization, using OT-guided diffusion sampling, IPC-aware label-image-aligned soft relabeling, and batch-wise Sinkhorn logit matching to reach state-of-the-art accuracy on ImageNet-1K and other datasets.
Problem
Large-scale dataset distillation methods match only global distributional statistics and miss instance-level and intra-class structure, leading to poor generalization of models trained on distilled sets.
Approach
Formulates dataset distillation as minimizing a Wasserstein/OT distance decomposed into three stages: Sinkhorn-OT guidance in latent diffusion sampling against real class latents, IPC-dependent teacher ensembles for soft labels, and batch-wise Sinkhorn logit matching for student training.
Key results
- On ImageNet-1K ResNet-18 at IPC=10, the method reaches 52.9% (300 epochs) and 58.6% (1000 epochs), above DiT-IGD at 45.5%.
- On ImageNette with ResNet-18 at IPC=10 under soft labels, the method reaches 79.0% versus 74.8% for DiT-IGD.
- The added OT overhead is reported as under 1% of sampling and matching time.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use OT-based dataset distillation | ImageNet-1K, ResNet-18 · ImageNet-1K, IPC=10 | top-1 accuracy: 45.5 -> 52.9 | Helped |
| use OT-based dataset distillation | ImageNet-100, ResNet-18 · ImageNet-100, IPC=10 | top-1 accuracy: 36.0 -> 47.7 | Helped |
| use OT-based dataset distillation | CIFAR-100, ConvNet-3 · CIFAR-100, IPC=10 | top-1 accuracy: 47.2$\pm$0.4 -> 50.7$\pm$0.2 | Helped |
| add OT-guided diffusion sampling (ablation) | ImageNette hard-label distillation · ImageNette, IPC=10, ConvNet-6 | top-1 accuracy: 61.9 -> 67.0 | Helped |
| add OT-guided diffusion sampling (ablation) | ImageNette hard-label distillation · ImageNette, IPC=10, ResNetAP-10 | top-1 accuracy: 66.5 -> 68.0 | Helped |
| add OT-guided diffusion sampling (ablation) | ImageNette hard-label distillation · ImageNette, IPC=10, ResNet-18 | top-1 accuracy: 67.7 -> 69.1 | Helped |
| add IPC-aware soft relabeling (ablation) | ImageNette soft-label distillation · ImageNette, IPC=10, ConvNet-6 | top-1 accuracy: 74.3 -> 74.5 | Helped |
| add IPC-aware soft relabeling (ablation) | ImageNette soft-label distillation · ImageNette, IPC=10, ResNetAP-10 | top-1 accuracy: 76.4 -> 77.8 | Helped |
| add IPC-aware soft relabeling (ablation) | ImageNette soft-label distillation · ImageNette, IPC=10, ResNet-18 | top-1 accuracy: 77.8 -> 79.0 | Helped |
| add OT-based student logit matching (ablation) | ImageNette soft-label distillation · ImageNette, IPC=10, ConvNet-6 | top-1 accuracy: 73.2 -> 74.5 | Helped |
| add OT-based student logit matching (ablation) | ImageNette soft-label distillation · ImageNette, IPC=10, ResNetAP-10 | top-1 accuracy: 75.9 -> 77.8 | Helped |
| add OT-based student logit matching (ablation) | ImageNette soft-label distillation · ImageNette, IPC=10, ResNet-18 | top-1 accuracy: 77.5 -> 79.0 | Helped |
Limitations
- OT hyperparameters (lambda, beta weights) are tuned per dataset in tables, and robustness is only shown on a limited grid.
- Relies on the IGD and EDC configurations and pretrained DiT/VAE, so gains are tied to those pipelines.
Experimental setup
Datasets: ImageNet-1K ImageNette ImageNet-100 CIFAR-100
Benchmarks: ImageNet-1K ImageNette ImageNet-100 CIFAR-100
Metrics: top-1 test accuracy runtime
Baselines: DM IDC DATM SRe2L RDED EDC DiT-IGD D4M
Builds on: IGD EDC Minimax Sinkhorn distance
Closest papers by meaning
- D^4: Dataset Distillation via Disentangled Diffusion Model (CVPR 2024 · similarity 0.88)
- Continuous-Time Distribution Matching for Few-Step Diffusion Distillation (NeurIPS 2026 · similarity 0.88)
- Dataset Distillation of 3D Point Clouds via Distribution Matching (NeurIPS 2025 · similarity 0.88)
- Improved Distribution Matching Distillation for Fast Image Synthesis (NeurIPS 2024 · similarity 0.87)
- Rectifying Soft-Label Entangled Bias in Long-Tailed Dataset Distillation (NeurIPS 2025 · similarity 0.87)
- DELT: A Simple Diversity-driven EarlyLate Training for Dataset Distillation (CVPR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.