Research map / Image generation / Fast diffusion sampling
SyncSDE: A Probabilistic Framework for Diffusion Synchronization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SyncSDE, a probabilistic framework that models inter-trajectory correlation as a conditional score term with a tunable λ, applied sequentially across diffusion synchronization tasks including mask-based T2I, image editing, wide image, ambiguous images, 3D texturing, and motion generation.
Problem
Diffusion synchronization methods that combine multiple diffusion trajectories rely on naive heuristics such as averaging without theoretical justification or task-specific adaptation, requiring extensive empirical search for each task.
Approach
Formulates synchronization as a conditional score function whose second term models the correlation between trajectories with a per-task probability (Gaussian with precision matrix and single hyperparameter λ), so that the design space of heuristics is reduced to choosing which correlation term to model.
Key results
- Mask-based T2I: SyncSDE (best) reaches KID 34.859 and FID 72.118 vs. MultiDiffusion 47.694 / 84.225 and SyncTweedies 117.360 / 149.470.
- Text-driven real image editing: SyncSDE (best) achieves LPIPS 0.254 and BG-LPIPS 0.222, better than DDIB, SDEdit, PTI, and MasaCtrl.
- Wide image generation: SyncSDE (best) lowers FID from 78.333 (SyncTweedies) to 44.707 and KID from 51.024 to 16.872.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use task-tuned SyncSDE synchronization instead of MultiDiffusion | Mask-based T2I with Stable Diffusion · Mask-based T2I | KID (×10³): 47.694 -> 34.859 | Helped |
| Use task-tuned SyncSDE synchronization instead of MultiDiffusion | Mask-based T2I with Stable Diffusion · Mask-based T2I | FID: 84.225 -> 72.118 | Helped |
| Use task-tuned SyncSDE synchronization instead of PTI | Text-driven real image editing with Stable Diffusion · Text-driven real image editing | CLIP-S: 0.322 -> 0.313 | Hurt |
| Use task-tuned SyncSDE synchronization instead of MasaCtrl | Text-driven real image editing with Stable Diffusion · Text-driven real image editing | LPIPS: 0.290 -> 0.254 | Helped |
| Use task-tuned SyncSDE synchronization instead of MasaCtrl | Text-driven real image editing with Stable Diffusion · Text-driven real image editing | BG-LPIPS: 0.341 -> 0.222 | Helped |
| Use SyncSDE instead of SyncTweedies | Wide image generation with Stable Diffusion · Wide image generation | KID (×10³): 51.024 -> 16.872 | Helped |
| Use SyncSDE instead of SyncTweedies | Wide image generation with Stable Diffusion · Wide image generation | FID: 78.333 -> 44.707 | Helped |
| Use SyncSDE instead of SyncTweedies | Wide image generation with Stable Diffusion · Wide image generation | CLIP-S: 0.328 -> 0.324 | Hurt |
| Use task-tuned SyncSDE instead of Visual Anagrams | Ambiguous image generation with DeepFloyd-IF · Ambiguous image generation | KID (×10³): 195.286 -> 174.902 | Helped |
| Use task-tuned SyncSDE instead of Visual Anagrams | Ambiguous image generation with DeepFloyd-IF · Ambiguous image generation | FID: 215.082 -> 208.788 | Helped |
| Use task-tuned SyncSDE instead of Visual Anagrams | Ambiguous image generation with DeepFloyd-IF · Ambiguous image generation | CLIP-S: 0.290 -> 0.272 | Hurt |
| Use SyncSDE instead of SyncMVD | 3D mesh texturing with ControlNet · 3D mesh texturing | KID (×10³): 196.341 -> 184.704 | Helped |
Surprising findings
- Setting 1/λ to 0 (λ infinite) makes the second trajectory independent, so it generates only the second prompt and fails to integrate both prompts.
- SyncTweedies' averaging produces blurry blends in ambiguous image generation, even though Visual Anagrams is designed specifically for that task and is outperformed by SyncSDE in FID.
Limitations
- λ must still be tuned per task for best results, and a fixed 1/λ=5 is used as a compromise.
- Evaluation relies on standard distribution and CLIP metrics without human studies.
Experimental setup
Datasets: LAION-5B
Benchmarks: LAION-5B subset (1,000 images) for editing custom wide image, mask-based T2I, ambiguous image, and 3D texturing sets
Metrics: KID FID CLIP-S LPIPS BG-LPIPS
Baselines: SyncTweedies MultiDiffusion Visual Anagrams SyncMVD DDIB SDEdit PTI MasaCtrl
Builds on: SyncTweedies Visual Anagrams MultiDiffusion DDIM conditional score estimation (CSG)
Closest papers by meaning
- How Distributed Collaboration Influences the Diffusion Model Training? A Theoretical Perspective (ICML 2025 · similarity 0.88)
- Test-time Alignment of Diffusion Models without Reward Over-optimization (ICLR 2025 · similarity 0.84)
- Resisting Stochastic Risks in Diffusion Planners with the Trajectory Aggregation Tree (ICML 2024 · similarity 0.84)
- Temporal Alignment Guidance: On-manifold Sampling in Diffusion Models (NeurIPS 2026 · similarity 0.84)
- Discrete-state Continuous-time Diffusion for Graph Generation (NeurIPS 2024 · similarity 0.84)
- Align Your Steps: Optimizing Sampling Schedules in Diffusion Models (ICML 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.