Research map / Image generation / Fast diffusion sampling
Unlocking Dataset Distillation with Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LD3M, which end-to-end optimizes distilled latent codes and class embeddings through a frozen pre-trained latent diffusion model using linearly decaying skip connections from the initial noisy latent, improving cross-architecture accuracy over GLaD on ImageNet subsets.
Problem
Dataset distillation with diffusion priors suffers from vanishing gradients when backpropagating through the long reverse denoising chain, so prior diffusion-based methods avoid gradient-based latent optimization and rely on sampling or GAN priors.
Approach
Inject a linearly decaying residual from the initial noisy latent z_T into every reverse diffusion step, creating a direct gradient path to the learnable latent Z without fine-tuning the diffusion weights, and initialize Z by encoding real images with the LDM autoencoder.
Key results
- On 128x128 ImageNet subsets at IPC=1, LD3M improves DC, MTT and DM by +3.76, +5.68 and +16.34 points over GLaD (per the table caption); the abstract reports up to 4.8 points.
- At IPC=10 on ImageNet A-E (128x128), LD3M improves average accuracy over GLaD by +2.5 (DC) and +3.5 (DM), reaching over 47%.
- On CIFAR-10 at IPC=1 with DM, LD3M reaches 24.9% average vs 23.8% for GLaD (trained G).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LD3M instead of GLaD with DC distillation | ImageNet subsets, 128×128, 1 IPC · ImageNet subsets, 128×128, 1 IPC (All), DC | accuracy: 29.8±1.3 -> 30.9±1.2 | Helped |
| Use LD3M instead of GLaD with DM distillation | ImageNet subsets, 128×128, 1 IPC · ImageNet subsets, 128×128, 1 IPC (All), DM | accuracy: 22.4±1.4 -> 25.9±1.2 | Helped |
| Use LD3M instead of GLaD with MTT distillation | ImageNet subsets, 128×128, 1 IPC · ImageNet subsets, 128×128, 1 IPC (All), MTT | accuracy: 29.0±1.2 -> 30.4±1.3 | Helped |
| Use LD3M instead of GLaD | ImageNet-A–E, 128×128, 10 IPC, DC · ImageNet-A–E, 128×128, 10 IPC, DC | accuracy: 45.9±1.0 -> 47.1±1.2 | Helped |
| Use LD3M instead of GLaD | ImageNet-A–E, 128×128, 10 IPC, DM · ImageNet-A–E, 128×128, 10 IPC, DM | accuracy: 45.8±0.6 -> 47.3±2.1 | Helped |
| Use LD3M instead of GLaD with random generator initialization | ImageNet-A–E, 256×256, 1 IPC, DC · ImageNet-A–E, 256×256, 1 IPC, DC | accuracy: 34.5±1.6 -> 36.5±1.6 | Helped |
| Add diffusion steps instead of using only the autoencoder (ablation) | LD3M, ImageNet-A–E, 128×128, 1 IPC, DC · ImageNet-A–E, 128×128, 1 IPC, DC | accuracy: 35.3±1.3 -> 36.5±1.3 | Helped |
| Add learnable latent codes to learnable conditioning (ablation) | LDM, ImageNette, 1 IPC, MTT · ImageNette, 1 IPC, MTT | average accuracy: 15.8±1.5 -> 22.3±2.0 | Helped |
| Add enhanced gradient flow to learnable conditioning and latent code (ablation) | LDM, ImageNette, 1 IPC, MTT · ImageNette, 1 IPC, MTT | average accuracy: 22.3±2.0 -> 28.1±3.3 | Helped |
| Initialize codes from real images rather than Gaussian noise (ablation) | LD3M, ImageNette, 1 IPC, MTT, 5K iterations · ImageNette, 1 IPC, MTT | average accuracy: 31.0±1.4 -> 32.0±1.3 | Helped |
| Initialize codes from real images rather than Gaussian noise (ablation) | LD3M, ImageNette, 1 IPC, DC, 5K iterations · ImageNette, 1 IPC, DC | average accuracy: 13.1±2.1 -> 32.9±2.1 | Helped |
| Initialize codes from real images rather than Gaussian noise (ablation) | LD3M, ImageNette, 1 IPC, DM, 5K iterations · ImageNette, 1 IPC, DM | average accuracy: 13.4±1.8 -> 26.8±1.7 | Helped |
Surprising findings
- Removing the diffusion steps (autoencoder-only) yields results comparable to GLaD (35.3% vs 35.4%), so the gain depends on optimizing through the reverse process.
- The modified reverse process produces more abstract outputs that deviate from the LDM's expected style when used for unconditional generation, with slightly reduced sample diversity.
Limitations
- The linear skip connection may not sufficiently counter vanishing gradients at larger diffusion step counts; performance deteriorates beyond about 40 steps.
- Modified reverse process no longer samples exactly from the LDM's data distribution and reduces sample diversity.
- Evaluated only on 10-class ImageNet subsets and CIFAR-10; no ImageNet-1K-scale results and no inter-class information used.
Experimental setup
Datasets: ImageNet-1k subsets (ImNet-A to E, ImNette, ImWoof, Birds, Fruits, Cats) CIFAR-10
Benchmarks: ImageNet subsets A-E, ImNette, ImWoof, Birds, Fruits, Cats at 128x128 (IPC=1, 10) ImageNet A-E at 256x256 (IPC=1) CIFAR-10 (IPC=1, 50)
Metrics: mean test accuracy standard deviation over 5 runs
Baselines: GLaD D4M Minimax Diffusion SRe2L Pixel space
Builds on: GLaD Dataset Condensation Distribution Matching Matching Training Trajectories Latent Diffusion Models
Closest papers by meaning
- Taming Diffusion for Dataset Distillation with High Representativeness (ICML 2025 · similarity 0.93)
- D^4: Dataset Distillation via Disentangled Diffusion Model (CVPR 2024 · similarity 0.92)
- MGD$^3$ : Mode-Guided Dataset Distillation using Diffusion Models (ICML 2025 · similarity 0.92)
- Efficient Dataset Distillation via Minimax Diffusion (CVPR 2024 · similarity 0.92)
- Plug-and-Play Diffusion Distillation (CVPR 2024 · similarity 0.90)
- Revisiting Diffusion Models: From Generative Pre-training to One-Step Generation (ICML 2025 · similarity 0.90)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.