Research map / Image generation / Text-to-image diffusion
StochSync: Stochastic Diffusion Synchronization for Image Generation in Arbitrary Spaces
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes StochSync, a zero-shot method that combines diffusion synchronization with score-distillation-style maximum stochasticity, multi-step x0 prediction, and non-overlapping views to generate images in arbitrary spaces such as 360° panoramas and mesh textures.
Problem
Zero-shot generation of images in arbitrary canonical spaces (e.g., panoramas, mesh textures) from a pretrained 2D image diffusion model; existing diffusion synchronization fails without strong conditioning and SDS lacks detail.
Approach
Interprets SDS as a DDIM refinement with maximum stochasticity and injects that into diffusion synchronization, replacing one-step Tweedie x0 prediction with multi-step denoising and sampling non-overlapping views while replacing gradient descent with exact l2 minimization over a linearly decreasing timestep schedule.
Key results
- On PanFusion prompts, StochSync reaches FID 57.88 and GIQA 20.30 (x10^3), the best among compared methods, versus 59.83 FID for L-MAGIC.
- Users preferred StochSync over L-MAGIC 56.20% on PanFusion prompts and 64.75% on horizon-specific prompts.
- On mesh texturing, StochSync gets FID 22.29 and KID 1.31, outperforming Paint3D, TEXTure and Text2Tex, and comparable to SyncTweedies (FID 21.76).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add maximum stochasticity (Max σₜ) to DS without multi-step x₀|t or non-overlapping views (ablation) | SyncTweedies-based panorama generation · Panorama generation (PanFusion prompts) | FID: 80.55 -> 138.82 | Hurt |
| Add multi-step x₀|t computation without Max σₜ or non-overlapping views (ablation) | SyncTweedies-based panorama generation · Panorama generation (PanFusion prompts) | FID: 80.55 -> 84.87 | Hurt |
| Add Max σₜ and multi-step x₀|t computation (ablation) | SyncTweedies-based panorama generation · Panorama generation (PanFusion prompts) | FID: 80.55 -> 78.56 | Helped |
| Add non-overlapping views to Max σₜ without multi-step x₀|t (ablation) | SyncTweedies-based panorama generation · Panorama generation (PanFusion prompts) | FID: 80.55 -> 117.09 | Hurt |
| Add multi-step x₀|t computation and non-overlapping views alongside Max σₜ (ablation) | StochSync · Panorama generation (PanFusion prompts) | FID: 80.55 -> 57.88 | Helped |
| Add multi-step x₀|t computation and non-overlapping views alongside Max σₜ (ablation) | StochSync · Panorama generation (PanFusion prompts) | IS: 8.65 -> 10.02 | Helped |
| Add multi-step x₀|t computation and non-overlapping views alongside Max σₜ (ablation) | StochSync · Panorama generation (PanFusion prompts) | GIQA: 18.22 -> 20.30 | Helped |
| Add multi-step x₀|t computation and non-overlapping views alongside Max σₜ (ablation) | StochSync · Panorama generation (PanFusion prompts) | CLIP: 30.07 -> 31.01 | Helped |
| Use StochSync instead of L-MAGIC | Panorama generation · Panorama generation (PanFusion prompts) | FID: 59.83 -> 57.88 | Helped |
| Use StochSync instead of MVDiffusion | Panorama generation · Panorama generation (PanFusion prompts) | IS: 10.87 -> 10.02 | Hurt |
| Use StochSync instead of SyncTweedies | 3D mesh texturing · StochSync | FID: 21.76 -> 22.29 | Hurt |
| Use StochSync instead of SyncTweedies | 3D mesh texturing · StochSync | KID: 1.46 -> 1.31 | Helped |
Surprising findings
- Maximum stochasticity alone degrades realism badly (FID rises from 80.55 to 138.82), and increasing denoising steps under maximum stochasticity can produce unrealistic images.
Limitations
- Synchronization overfits to individual views when applied to NeRF or Gaussian splat generation.
- Mesh texturing is designed for the conditional case and only comparable to, not better than, SyncTweedies.
- Maximum stochasticity trades coherence against realism.
Experimental setup
Datasets: PanFusion prompts L-MAGIC prompts
Benchmarks: PanFusion prompts L-MAGIC prompts SyncTweedies mesh-prompt set (429 pairs)
Metrics: FID IS GIQA CLIP score KID user preference
Baselines: SDS SDI ISM MVDiffusion PanFusion L-MAGIC SyncTweedies Paint-it Paint3D TEXTure Text2Tex
Builds on: SyncTweedies DDIM Score Distillation Sampling DreamTime SDEdit
Closest papers by meaning
- SyncTweedies: A General Generative Framework Based on Synchronized Diffusions (NeurIPS 2024 · similarity 0.90)
- Taming Stable Diffusion for Text to 360 Panorama Image Generation (CVPR 2024 · similarity 0.87)
- DreamTime: An Improved Optimization Strategy for Diffusion-Guided 3D Generation (ICLR 2024 · similarity 0.87)
- MotionCraft: Physics-Based Zero-Shot Video Generation (NeurIPS 2024 · similarity 0.87)
- STDD: Spatio-Temporal Dual Diffusion for Video Generation (CVPR 2025 · similarity 0.87)
- Single Mesh Diffusion Models with Field Latents for Texture Generation (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.