Research map / Image generation / Fast diffusion sampling
ResMaster: Mastering High-Resolution Image Generation via Structural and Fine-Grained Guidance
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ResMaster, a training-free patch-based method that upscales SDXL images to 4K-8K using low-frequency structural guidance from a low-resolution reference plus IP-Adapter image prompts and VLM-generated patch captions.
Problem
Diffusion models trained at about 1024x1024 produce structural distortions, repetitive patterns and over-smoothed content when generating 4K or higher resolution images without retraining.
Approach
Replaces the low-frequency components of each high-resolution patch's predicted clean latent with those of a low-resolution reference at every denoising step (FFT-based swapping, instead of gradient guidance), and adds per-patch image prompts (IP-Adapter from the low-res patch) and dense per-patch VLM captions for fine-grained guidance.
Key results
- At 4096x4096, ResMaster reports FID_r 65.43 and CLIP 30.95, best among compared methods, versus DemoFusion FID_r 76.03 and CLIP 30.83.
- At 4096x4096, ResMaster takes about 6 minutes versus 11 minutes for DemoFusion on a single A100.
- Ablation at 4096x4096: the base MultiDiffusion FID_r of 92.80 drops to 65.43 with all three components (SG, TIP, ETP).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add Structural Guidance (SG) to Multi-Diffusion (ablation) | Multi-Diffusion · Laion-5B, 4096×4096 | FID_r: 92.80 -> 79.43 | Helped |
| Add Structural Guidance (SG) to Multi-Diffusion (ablation) | Multi-Diffusion · Laion-5B, 4096×4096 | CLIP: 27.19 -> 28.39 | Helped |
| Add Tailored Image Prompts (TIP) to SG (ablation) | Multi-Diffusion + SG · Laion-5B, 4096×4096 | FID_r: 79.43 -> 66.36 | Helped |
| Add Tailored Image Prompts (TIP) to SG (ablation) | Multi-Diffusion + SG · Laion-5B, 4096×4096 | CLIP: 28.39 -> 30.28 | Helped |
| Add Enriched Textual Prompts (ETP) to SG and TIP (ablation) | Multi-Diffusion + SG + TIP · Laion-5B, 4096×4096 | FID_r: 66.36 -> 65.43 | Helped |
| Add Enriched Textual Prompts (ETP) to SG and TIP (ablation) | Multi-Diffusion + SG + TIP · Laion-5B, 4096×4096 | CLIP: 30.28 -> 30.95 | Helped |
| Provide the full image description as VLM context when recaptioning patches (ablation) | ResMaster · Image patches | visual quality | No clear effect |
Surprising findings
- Providing the full image description as context to the VLM did not help and could induce hallucinations, so per-patch captions were used instead.
Limitations
- Performance is bounded by the quality of the pre-trained base model and the low-resolution reference image.
- Generating higher resolutions requires processing many patches, leading to long generation times.
- Compared to DemoFusion at 4096x4096, FID_c is worse (55.09 vs 56.75 is better for ResMaster, but SDXL+BSRGAN is best at 50.89), so not best on all metrics.
Experimental setup
Datasets: LAION-5B
Benchmarks: LAION-5B subset (1K captions, 10K real images)
Metrics: FID IS CLIP score inference time
Baselines: SDXL direct inference SDXL + BSRGAN ScaleCrafter DemoFusion MultiDiffusion SyncDiffusion
Builds on: MultiDiffusion SDXL IP-Adapter DemoFusion ScaleCrafter
Closest papers by meaning
- DiffuseHigh: Training-Free Progressive High-Resolution Image Synthesis Through Structure Guidance (AAAI 2025 · similarity 0.90)
- ScaleCrafter: Tuning-free Higher-Resolution Visual Generation with Diffusion Models (ICLR 2024 · similarity 0.90)
- RepLDM: Reprogramming Pretrained Latent Diffusion Models for High-Quality, High-Efficiency, High-Resolution Image Generation (NeurIPS 2025 · similarity 0.89)
- FAM Diffusion: Frequency and Attention Modulation for High-Resolution Image Generation with Stable Diffusion (CVPR 2025 · similarity 0.89)
- ResAdapter: Domain Consistent Resolution Adapter for Diffusion Models (AAAI 2025 · similarity 0.89)
- UltraPixel: Advancing Ultra High-Resolution Image Synthesis to New Peaks (NeurIPS 2024 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.