Research map / Image generation / Fast diffusion sampling
Few-Shot Diffusion Models Escape the Curse of Dimensionality
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Theoretical analysis of few-shot diffusion models under a linear latent structure, showing fine-tuning only the linear encoder/decoder escapes the curse of dimensionality for approximation and admits a closed-form minimizer in a Gaussian latent case.
Problem
Few-shot diffusion models fine-tune a pretrained diffusion model on 5-10 target samples, but no theory explains their small approximation error, since prior bounds without fine-tuning suffer the curse of (latent) dimensionality.
Approach
Under a shared linear latent structure, freeze the pretrained latent network and fine-tune only the linear encoder/decoder V on target data; this yields an approximation bound Õ(n_s^{-2/d} + n_ta^{-1/2}) that depends on n_ta only through a dimension-free-in-d term, and in the Gaussian latent case the few-shot objective has a closed-form minimizer.
Key results
- Few-shot models obtain Õ(n_s^{-2/d} + n_ta^{-1/2}) approximation bound on the score, improving on n_ta^{-2/d} without fine-tuning.
- In the Gaussian latent case the expected few-shot objective has a closed-form minimizer, with empirical subspace error Õ(1/n_ta + 1/(δ√n_s)).
- Fine-tuning only encoder/decoder layers with 10 bald target images produced novel bald faces, while full fine-tuning overfit and only reproduced target-like images with brightness/angle changes.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Fine-tune the target encoder and decoder while reusing the pretrained latent network | Few-shot diffusion models · Linear-structure distributions | score approximation error bound: eO(n−2/d_s + n−1/2_ta) vs. n−2/d_ta | Helped |
| Use the empirical closed-form minimizer | Few-shot diffusion models with Gaussian latent distribution · Gaussian latent distribution | accuracy bound: eO(1/nta + 1/√ns) | Helped |
| Fine-tune encoder and decoder layers only rather than all model parameters (ablation) | CelebA64 diffusion model · CelebA64 bald-feature generation | generalization to novel images with the target feature | Helped |
| Fine-tune all model parameters instead of encoder and decoder layers only (ablation) | CelebA64 diffusion model · CelebA64 bald-feature generation | generalization to novel images with the target feature | Hurt |
Surprising findings
- Full fine-tuning on 10 target images mainly memorizes the target set rather than generalizing, while encoder/decoder-only fine-tuning generalizes.
Limitations
- Analysis assumes a linear low-dimensional structure, which is less general than bounded distributions.
- Closed-form result only holds for a Gaussian latent distribution; general latents need optimization guarantees.
- Experiments are small-scale qualitative checks on CelebA64 and cat faces without quantitative metrics.
Experimental setup
Datasets: CelebA64 cat face dataset
Benchmarks: CelebA64 (qualitative) cat face (qualitative)
Metrics: qualitative visual inspection of generated samples
Baselines: full fine-tuning (all parameters) n_ta^{-2/d} bound without fine-tuning
Builds on: Chen et al. 2023 score approximation for linear structure distributions Yuan et al. 2023 Gaussian latent diffusion analysis DreamBooth LoRA
Closest papers by meaning
- Adapting to Unknown Low-Dimensional Structures in Score-Based Diffusion Models (NeurIPS 2024 · similarity 0.89)
- Few-shot Learner Parameterization by Diffusion Time-steps (CVPR 2024 · similarity 0.88)
- Theory on Score-Mismatched Diffusion Models and Zero-Shot Conditional Samplers (ICLR 2025 · similarity 0.87)
- Simpler Diffusion: 1.5 FID on ImageNet512 with Pixel-space Diffusion (CVPR 2025 · similarity 0.87)
- Plug-and-Play Diffusion Distillation (CVPR 2024 · similarity 0.87)
- Training Diffusion-based Generative Models with Limited Data (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.