Research map / Image generation / Editing and style transfer
Guided and Variance-Corrected Fusion with One-shot Style Alignment for Large-Content Image Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Guided Fusion, Variance-Corrected Fusion, and one-shot Style Alignment to reduce seams, blur, and style inconsistency when jointly denoising overlapping patches of small diffusion models for large-content (panorama) image generation.
Problem
Large-content image generation with small pretrained diffusion models via joint denoising of overlapping patches suffers from seams, inconsistent objects and styles, and blurring (especially with SDE samplers like DDPM) due to averaging overlapped regions.
Approach
Weight overlapped patch values by a center-biased guidance map (Guided Fusion), rescale the averaged noise to restore the variance lost by averaging Gaussian samples (Variance-Corrected Fusion), and once-only spherical interpolation of non-overlapping initial noise patches toward a reference noise (Style Alignment).
Key results
- GF with DDIM improves FID from 11.22 (MultiDiffusion) to 10.71 on the 10-prompt panorama evaluation (stride 384 setting, Table 2).
- DDPM with VCF reaches FID 4.78 versus 11.22 for DDIM-based MultiDiffusion, and VCF+GF+SA0.4 reaches FID 4.02.
- SA with alpha=0.4 reduces single-image runtime cost versus SyncDiffusion: about 8s versus 102s for a 3584-width image at stride 384 on a Quadro RTX 6000.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace uniform averaging with guided weighted fusion | MultiDiffusion · Text-to-panorama with DDIM | FID: 11.22 -> 10.71 | Helped |
| Add guided weighted fusion | MultiDiffusion · Text-to-panorama with DDIM, stride 128 | FID: 20.60 -> 17.64 | Helped |
| Add guided weighted fusion | MultiDiffusion · Text-to-panorama with DDIM, stride 256 | FID: 17.32 -> 15.99 | Helped |
| Add guided weighted fusion | MultiDiffusion · Text-to-panorama with DDIM, stride 384 | FID: 15.55 -> 14.88 | Helped |
| Add variance-corrected fusion for DDPM sampling | MultiDiffusion-style patch fusion · Text-to-panorama with DDPM | FID | Helped |
| Add guided fusion to VCF (ablation) | VCF · Text-to-panorama with DDPM | FID: 4.78 -> 4.35 | Helped |
| Add Style Alignment with α=0.4 (ablation) | GF · Text-to-panorama with DDIM | FID: 10.71 -> 10.40 | Helped |
| Add Style Alignment with α=0.4 (ablation) | GF · Text-to-panorama with DDIM | GIQA-DS: 8.887 -> 8.877 | No clear effect |
| Increase Style Alignment α (ablation) | MD and GF · Text-to-panorama with DDIM | FID | Helped |
| Increase Style Alignment α (ablation) | MD and GF · Text-to-panorama with DDIM | GIQA-DS | Hurt |
| Use Style Alignment with α larger than 0.4 (ablation) | MD and GF · Text-to-panorama with DDIM | GIQA-DS | Hurt |
| Use one-shot Style Alignment rather than SyncDiffusion’s iterative alignment | Style consistency alignment · Text-to-panorama with DDIM | runtime: 102 seconds -> approximately 8s | Helped |
Surprising findings
- Averaging overlapped patches with DDPM (SDE sampler) produces highly blurred images, so MultiDiffusion+DDPM was not reported.
- SA results with alpha=0.1 are highly similar to SyncDiffusion (SSIM 0.74), suggesting the two methods converge to similar content.
Limitations
- VCF with DDPM requires 1000 denoising steps, making runtime about 181s per image.
- Style Alignment trades diversity for consistency; diversity drops quickly for alpha above 0.4.
- Evaluation relies on images from Stable Diffusion v2.0 as a reference set, and on a limited set of ten prompts.
Experimental setup
Benchmarks: text-to-panorama generation (10 prompts, 512x3584 images)
Metrics: FID KID GIQA-QS GIQA-DS CLIP score SSIM runtime
Baselines: MultiDiffusion SyncDiffusion ElasticDiffusion
Builds on: MultiDiffusion SyncDiffusion DDPM DDIM Stable Diffusion
Closest papers by meaning
- Patched Denoising Diffusion Models For High-Resolution Image Synthesis (ICLR 2024 · similarity 0.85)
- Conditional Controllable Image Fusion (NeurIPS 2024 · similarity 0.85)
- High-Fidelity Diffusion-Based Image Editing (AAAI 2024 · similarity 0.84)
- Energy-Guided Optimization for Personalized Image Editing with Pretrained Text-to-Image Diffusion Models (AAAI 2025 · similarity 0.84)
- TiNO-Edit: Timestep and Noise Optimization for Robust Diffusion-Based Image Editing (CVPR 2024 · similarity 0.84)
- ZoomLDM: Latent Diffusion Model for Multi-scale Image Generation (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.