Research map / Image generation / Fast diffusion sampling
Continuous Semi-Implicit Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CoSIM, a continuous-time hierarchical semi-implicit model whose continuous transition kernel is trained with SIVI/score-identity objectives (plus a regularization shift on the auxiliary function) to give simulation-free multistep distillation of diffusion models.
Problem
Hierarchical semi-implicit variational inference for diffusion distillation converges slowly because its layers are trained sequentially over a fixed discrete time sequence.
Approach
Replace the discrete hierarchy with a continuous transition kernel q(x_s|x_t;s,t) that maps the mixing distribution at any time t to the target at time s, trained on sampled (s,t) pairs via a SiD-style score-identity objective, with a regularized auxiliary function shifted toward the pretrained score network and a consistency-model parameterization.
Key results
- On CIFAR-10 unconditional generation, CoSIM with 4 NFE reaches FID 1.97 and FD-DINOv2 113.51, outperforming PFGM++ (141.65) on FD-DINOv2.
- On ImageNet 64x64, CoSIM with 4 NFE achieves FID 1.46 and FD-DINOv2 58.66 at 296M parameters.
- On ImageNet 512x512, CoSIM-L with 4 NFE reaches FD-DINOv2 41.79, beating EDM2-XXL-DINOv2 (42.84) at 63 NFE.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Increase sampling from 2 to 4 NFE | CoSIM · CIFAR-10 (32×32) | FID: 2.40 -> 1.97 | Helped |
| Increase sampling from 2 to 4 NFE | CoSIM · CIFAR-10 (32×32) | FD-DINOv2: 116.92 -> 113.51 | Helped |
| Increase sampling from 2 to 4 NFE | CoSIM · ImageNet (64×64) | FID: 3.35 -> 1.46 | Helped |
| Increase sampling from 2 to 4 NFE | CoSIM · ImageNet (64×64) | FD-DINOv2: 108.99 -> 58.66 | Helped |
| Increase sampling from 2 to 4 NFE | CoSIM-M · ImageNet (512×512) | FD-DINOv2: 53.35 -> 51.57 | Helped |
| Increase sampling from 2 to 4 NFE | CoSIM-L · ImageNet (512×512) | FD-DINOv2: 46.41 -> 41.79 | Helped |
| Increase sampling from 2 to 4 NFE | CoSIM-S · ImageNet (512×512) | FD-DINOv2: 67.81 -> 67.71 | Helped |
| Increase sampling from 2 to 4 NFE | CoSIM-L · ImageNet (512×512) | FID: 1.84 -> 1.83 | Helped |
| Use CoSIM rather than the teacher at 63 NFE | CoSIM-L, 4 NFE · ImageNet (512×512) | FD-DINOv2: 42.84 -> 41.79 | Helped |
| Use CoSIM rather than the teacher at 63 NFE | CoSIM-M, 2 NFE · ImageNet (512×512) | FD-DINOv2: 58.44 -> 53.35 | Helped |
Surprising findings
- CoSIM with 2 NFE on ImageNet 512x512 (CoSIM-L, FD-DINOv2 46.41) already surpasses the 63-NFE EDM2-XL teacher on FD-DINOv2.
- FID and FD-DINOv2 rank methods differently; CTM wins FID on CIFAR-10 while CoSIM wins FD-DINOv2.
Limitations
- High memory consumption since training involves three models (generator, auxiliary function, target score).
- Initialization from pretrained score limits use of larger architectures for the transition kernel.
- One-step generation quality lags modern one-step distillation methods such as SiD and sCD.
Experimental setup
Datasets: CIFAR-10 ImageNet
Benchmarks: CIFAR-10 ImageNet 64x64 ImageNet 512x512
Metrics: FID FD-DINOv2
Baselines: HSIVI-SM CD-LPIPS iCT sCT CTM ECT SiD Moment Matching PFGM++ Diff-Instruct EDM2 sCD
Builds on: HSIVI SiD consistency models EDM SIVI
Closest papers by meaning
- One-Step Diffusion Distillation through Score Implicit Matching (NeurIPS 2024 · similarity 0.87)
- Kernel Semi-Implicit Variational Inference (ICML 2024 · similarity 0.86)
- CacheQuant: Comprehensively Accelerated Diffusion Models (CVPR 2025 · similarity 0.86)
- Simplifying, Stabilizing and Scaling Continuous-time Consistency Models (ICLR 2025 · similarity 0.86)
- On improved Conditioning Mechanisms and Pre-training Strategies for Diffusion Models (NeurIPS 2024 · similarity 0.86)
- SCott: Accelerating Diffusion Models with Stochastic Consistency Distillation (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.