Research map / Image generation / Fast diffusion sampling
Generalized Consistency Trajectory Models for Image Manipulation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes generalized consistency trajectory models (GCTMs) that learn one-step ODE translation between arbitrary distributions via flow matching, and applies them to image-to-image translation, restoration, editing and latent manipulation.
Problem
Consistency trajectory models only translate from Gaussian noise to data, limiting their use for fast guided image manipulation and image-to-image translation that requires transport between arbitrary distributions.
Approach
Generalizes CTM's PFODE trajectory learning to flow matching between two arbitrary distributions with a design space of couplings (independent, entropy-regularized OT via Sinkhorn-Knopp, supervised) plus Gaussian perturbation of x1; CTM is shown to be a special case when one side is Gaussian.
Key results
- On CIFAR10 with NFE=1 and no teacher, GCTM (OT coupling) reaches FID 5.32, outperforming CM (8.70) and CTM (9.00) without teacher, and nearly matching CTM with teacher (5.28); iCM is better at 2.51.
- On Edges→Shoes image-to-image translation at NFE=1, GCTM achieves FID 40.3 and LPIPS 0.097, beating Pix2Pix, Palette and I2SB baselines run at 5 NFE.
- In zero-shot FFHQ 64x64 restoration at 32 NFE, GCTM gets PSNR 31.61 on SR2 bicubic versus 31.19 for DPS and 30.80 for CM.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use OT coupling instead of independent coupling for unconditional generation | GCTM · CIFAR10 unconditional generation | training iterations to reach a performance level: ×2.5 | Helped |
| Use OT coupling instead of independent coupling | GCTM · CIFAR10 unconditional generation | FID: 24.7 -> 18.2 (N = 4) | Helped |
| Use GCTM instead of Pix2Pix | image-to-image translation · Edges→Shoes | FID: 77.0 -> 40.3 | Helped |
| Use GCTM instead of I²SB | image-to-image translation · Night→Day | FID: 145.8 -> 148.8 | No clear effect |
| Use GCTM instead of regression | supervised image restoration · FFHQ, SR2 - Bicubic | PSNR: 33.46 -> 32.37 | Hurt |
| Use GCTM instead of CM | zero-shot image restoration · FFHQ, SR2 - Bicubic | PSNR: 30.80 -> 31.61 | Helped |
| Use Gaussian perturbation on the input distribution (ablation) | GCTM image-to-image translation · Edges→Shoes | FID | Helped |
| Set σ_max to 500 instead of 80, with Gaussian perturbation (ablation) | GCTM image-to-image translation · Edges→Shoes | FID convergence speed | Helped |
| Train without Gaussian perturbation and use σ_max = 80 (ablation) | GCTM image-to-image translation · Edges→Shoes | FID: 30 | Hurt |
Surprising findings
- Plain regression with l2 loss gives the best PSNR/SSIM in supervised restoration, outperforming GCTM on distortion metrics.
- CM-based zero-shot restoration is slightly worse than DPS, attributed to error accumulation at large NFE.
Limitations
- Not yet state-of-the-art on unconditional generation.
- Requires Gaussian perturbation and large sigma_max for image-to-image settings; regression yields better distortion metrics in supervised restoration.
Experimental setup
Datasets: CIFAR10 Edges→Shoes Night→Day Facades FFHQ ImageNet
Benchmarks: CIFAR10 Edges→Shoes Night→Day Facades FFHQ ImageNet
Metrics: FID IS LPIPS PSNR SSIM inference time (ms)
Baselines: CTM CM iCM Pix2Pix Palette I2SB DPS BBDM EGSDE Regression
Builds on: consistency trajectory models flow matching consistency models EDM iCT
Closest papers by meaning
- Consistency Trajectory Models: Learning Probability Flow ODE Trajectory of Diffusion (ICLR 2024 · similarity 0.90)
- Consistency Diffusion Bridge Models (NeurIPS 2024 · similarity 0.88)
- On the Trajectory Regularity of ODE-based Diffusion Sampling (ICML 2024 · similarity 0.88)
- Simplifying, Stabilizing and Scaling Continuous-time Consistency Models (ICLR 2025 · similarity 0.87)
- High-Fidelity Diffusion-Based Image Editing (AAAI 2024 · similarity 0.87)
- Truncated Consistency Models (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.