Research map / Image generation / Fast diffusion sampling
VCT: Training Consistency Models with Variational Noise Coupling
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Variational Consistency Training (VCT), which learns a data-dependent Gaussian noise coupling via an encoder regularized by a KL term (VAE-style) and plugs it into iCT and ECM consistency training.
Problem
Non-distillation consistency training suffers from high gradient variance and instability because fixed independent noise-data pairings create singular, hard-to-learn ODE flows.
Approach
Learn a data-conditional noise coupling q_phi(x1|x0) with an encoder (mean and scale networks) trained jointly with the consistency model, regularized by a KL term to the standard Gaussian prior (weighted by beta), giving an ELBO-like objective compatible with VE and linear-interpolation kernels.
Key results
- VCT (iCT-LI-VC) reaches 3.62 1-step / 2.22 2-step FID on Fashion-MNIST, improving over iCT-LI (4.75 / 3.46).
- ECM-LI-VC reaches 4.93 1-step and 3.07 2-step FID on ImageNet 64x64 (200k iterations), versus 5.13 / 3.20 for ECM-LI.
- ECM-VE-VC reaches 3.26 1-step / 2.02 2-step FID on CIFAR-10, the best non-distillation CT result reported in the paper's tables.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add variational coupling to iCT-LI | iCT · Fashion-MNIST | FID: 4.75 / 3.46 -> 3.62 / 2.22 | Helped |
| Add variational coupling to iCT-VE | iCT · CIFAR-10 | FID: 3.61 / 2.79 -> 2.86 / 2.32 | Helped |
| Add variational coupling to ECM-VE | ECM · CIFAR-10 | FID: 3.68 / 2.14 -> 3.26 / 2.02 | Helped |
| Add variational coupling to ECM-VE | ECM · FFHQ 64×64 | FID: 5.99 / 4.39 -> 5.47 / 4.16 | Helped |
| Add variational coupling to ECM-LI | ECM · ImageNet 64×64, 200k iterations | FID: 5.13 / 3.20 -> 4.93 / 3.07 | Helped |
| Add variational coupling to ECM-VE | ECM · ImageNet 64×64, 200k iterations | FID: 5.26 / 3.22 -> 5.08 / 3.15 | Helped |
| Use variational coupling instead of independent noise | iCT-VE · CIFAR-10, gradient variance during training | gradient variance | Mixed |
| Use β=5 (ablation) | iCT-VE-VC · Fashion-MNIST, FID | FID: 3.88 / 2.37 -> 12.53 / 5.69 | Hurt |
| Use β=15 instead of β=30 (ablation) | iCT-VE-VC · Fashion-MNIST, 1-step FID | FID: 3.88 -> 4.97 | Hurt |
| Use β=15 instead of β=30 (ablation) | iCT-VE-VC · Fashion-MNIST, 2-step FID | FID: 2.37 -> 2.34 | No clear effect |
| Use β=60 instead of β=30 (ablation) | iCT-VE-VC · Fashion-MNIST, FID | FID: 3.88 / 2.37 -> 3.90 / 2.67 | Hurt |
| Train for 100k instead of 200k iterations (ablation) | ECM with variational coupling · ImageNet 64×64 | FID: ECM-VE: 5.26 / 3.22 -> 5.67 / 3.67; ECM-LI: 5.13 / 3.20 -> 6.34 / 3.77 | Mixed |
Surprising findings
- Learned coupling shows higher gradient variance early in training, lower variance only later as the encoder converges.
- The original iCT reports a better 1-step CIFAR-10 FID (2.83) than the VCT re-implementation.
Limitations
- Requires tuning the KL weight beta, which varies across datasets and kernels.
- Learned coupling needs more iterations to converge; at 100k ImageNet iterations VCT is similar or worse than baselines.
- Gradient clipping is needed for stability with learned coupling.
Experimental setup
Datasets: Fashion-MNIST CIFAR-10 FFHQ 64x64 ImageNet 64x64
Benchmarks: Fashion-MNIST CIFAR-10 FFHQ 64x64 ImageNet 64x64
Metrics: FID gradient variance
Baselines: iCT ECM iCT-OT ECM-OT iCT (original reported) ECM (original reported)
Builds on: improved consistency training easy consistency models variational autoencoder flow matching minibatch optimal transport coupling
Closest papers by meaning
- Truncated Consistency Models (ICLR 2025 · similarity 0.88)
- Improved Techniques for Training Consistency Models (ICLR 2024 · similarity 0.88)
- Improved Training Technique for Latent Consistency Models (ICLR 2025 · similarity 0.88)
- Adaptive Discretization for Consistency Models (NeurIPS 2025 · similarity 0.88)
- Improving Consistency Models with Generator-Augmented Flows (ICML 2025 · similarity 0.86)
- Enhancing Consistency-Based Image Generation via Adversarialy-Trained Classification and Energy-Based Discrimination (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.