Research map / Image generation / Fast diffusion sampling
Towards Universal Dataset Distillation via Task-Driven Diffusion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes UniDD, a task-driven diffusion framework that uses frozen task-specific proxy models (classifier, detector, segmenter) to guide Stable Diffusion sampling for dataset distillation across classification, detection, and segmentation.
Problem
Dataset distillation has focused on image classification; extending it to object detection and segmentation is hard because of heterogeneous task objectives and inflexible global diffusion generation for multi-object images.
Approach
Train task-specific proxy models on real data and use their task loss on the predicted clean image at each diffusion sampling step to steer Stable Diffusion toward target labels (classes, boxes, masks), with VLM-generated task-aligned prompts and proxy-based filtering and relabeling, without fine-tuning the diffusion model.
Key results
- On ImageNet-1K at IPC-10, UniDD reaches 50.4% top-1 accuracy with ResNet-18, 6.1 points above the prior best diffusion method MiniMax Diffusion (44.3%).
- On Pascal VOC detection at 1% compression, UniDD achieves 16.8 mAP versus 5.7 for the best coreset baseline (Uniform).
- On Pascal VOC segmentation at 14% ratio, UniDD reaches 14.9 mIoU versus 12.1 for the best coreset baseline.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use UniDD instead of MiMxDiff at IPC-10 | ImageNet-1K dataset distillation · ImageNet-1K, IPC-10 | accuracy: 44.3 -> 50.4 | Helped |
| Add task-specific proxy guidance to the diffusion baseline (ablation) | UniDD · ImageNette, IPC-10 | accuracy: 60.4±0.9 -> 64.2±0.8 | Helped |
| Add task-specific proxy guidance to the diffusion baseline (ablation) | UniDD · Pascal VOC detection, 1% ratio | mAP: 11.5±0.2 -> 14.7±0.4 | Helped |
| Add task-specific proxy guidance to the diffusion baseline (ablation) | UniDD · Pascal VOC segmentation, 3.5% ratio | mIoU: 7.2±0.3 -> 8.9±0.5 | Helped |
| Add task-aligned contextual prompting to task-driven diffusion (ablation) | UniDD · ImageNette, IPC-10 | accuracy: 64.2±0.8 -> 64.7±0.4 | Helped |
| Add task-aligned contextual prompting to task-driven diffusion (ablation) | UniDD · Pascal VOC detection, 1% ratio | mAP: 14.7±0.4 -> 15.9±0.3 | Helped |
| Add task-aligned contextual prompting to task-driven diffusion (ablation) | UniDD · Pascal VOC segmentation, 3.5% ratio | mIoU: 8.9±0.5 -> 9.6±0.4 | Helped |
| Add proxy-driven high-realism refinement (ablation) | UniDD · ImageNette, IPC-10 | accuracy: 64.7±0.4 -> 66.0±0.7 | Helped |
| Add proxy-driven high-realism refinement (ablation) | UniDD · Pascal VOC detection, 1% ratio | mAP: 15.9±0.3 -> 16.8±0.5 | Helped |
| Add proxy-driven high-realism refinement (ablation) | UniDD · Pascal VOC segmentation, 3.5% ratio | mIoU: 9.6±0.4 -> 10.3±0.5 | Helped |
| Use a more complex TSP model (ResNet-101 instead of ResNet-18) (ablation) | UniDD synthetic data · ImageNette, ResNet-18 test model, IPC-10 | accuracy: 66.0±0.7 -> 69.5±0.7 | Helped |
| Increase guidance strength beyond λs = 100 (ablation) | UniDD · ImageWoof and ImageNette, IPC-10 | accuracy | Hurt |
Surprising findings
- Cross-architecture results show no degradation, unlike optimization-based distillation methods, when test architecture differs from the TSP model.
Limitations
- Relies on SD v1.5 and a VLM (GPT-4o) for prompts, adding dependency on proprietary components.
- Detection and segmentation baselines are only coreset methods, so no direct comparison with prior DD for these tasks.
Experimental setup
Datasets: ImageNet-1K ImageWoof ImageNette ImageFruit ImageMeow Pascal VOC MS COCO
Benchmarks: ImageNet-1K ImageNet-Subset (ImageWoof, ImageFruit, ImageNette, ImageMeow) Pascal VOC MS COCO
Metrics: top-1 accuracy mAP AP50 mIoU
Baselines: TESLA SRe2L D4M RDED MiniMax Diffusion DM MTT HaBa DataDAM Random Uniform K-Center
Builds on: Stable Diffusion v1.5 DDIM classifier guidance D4M RDED
Closest papers by meaning
- $\mathbf{D^{3}S^{2}}$: Diffusion-Guided Dataset Distillation for Semantic Segmentation (NeurIPS 2026 · similarity 0.88)
- Diff-Plugin: Revitalizing Details for Diffusion-based Low-level Tasks (CVPR 2024 · similarity 0.88)
- D^4: Dataset Distillation via Disentangled Diffusion Model (CVPR 2024 · similarity 0.88)
- Unlocking Dataset Distillation with Diffusion Models (NeurIPS 2025 · similarity 0.88)
- Efficient Dataset Distillation via Minimax Diffusion (CVPR 2024 · similarity 0.87)
- MGD$^3$ : Mode-Guided Dataset Distillation using Diffusion Models (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.