Research map / Image generation / Autoregressive and latent generation
Is One GPU Enough? Pushing Image Generation at Higher-Resolutions with Foundation Models.
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Pixelsmith is a zero-shot, single-GPU framework that scales SDXL outputs to very high resolutions by cascading from a base image, using a Slider-controlled FFT phase-guidance from an upsampled image plus patch-wise denoising with chess-mask blending.
Problem
Generating images far beyond the native resolution of pre-trained text-to-image diffusion models (e.g., 1024^2) causes duplications and artifacts, and existing high-resolution methods need expensive GPUs, long sampling times, or retraining.
Approach
Combine random patch denoising (adapted from DiffInfinite) with a guidance latent from a Lanczos-upsampled base image; fuse phases (averaged) with the current amplitudes via FFT, control the guidance strength with a tunable Slider step, and blend with a chess-like mask to suppress prompt-induced duplications.
Key results
- At 2048^2, Pixelsmith reaches FID 63.116 and IS 24.242 versus DemoFusion's 64.422 FID and 19.307 IS, with 130 s sampling time on a single RTX 3090.
- At 4096^2, Pixelsmith achieves FID 63.686 and CLIP 32.369, beating DemoFusion (66.186 FID) and HiDiffusion, but runs in 549 s versus HiDiffusion's 255 s.
- The method reports a 1000x scale-up (gigapixel output) with 8.4 GB memory on a single RTX 3090 across tested resolutions.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Pixelsmith instead of DemoFusion | SDXL-based higher-resolution generation · 2048² | FID: 64.422 -> 63.116 | Helped |
| Use Pixelsmith instead of DemoFusion | SDXL-based higher-resolution generation · 4096² | FID: 66.186 -> 63.686 | Helped |
| Use Pixelsmith instead of the 2× upscaler | 2048² image generation · 2048² | FID: 63.913 -> 63.116 | Helped |
| Change Slider position from 1 to 30 (ablation) | Pixelsmith · 2048² | FID: 74.890 -> 63.116 | Helped |
| Change Slider position from 24 to 30 (ablation) | Pixelsmith · 2048² | FID: 63.168 -> 63.116 | Helped |
| Change Slider position from 49 to 30 (ablation) | Pixelsmith · 2048² | FID: 64.195 -> 63.116 | Helped |
| Use current-latent amplitude and averaged guidance/current phase instead of averaging amplitudes (ablation) | Pixelsmith · 2048² | FID: 63.476 -> 63.116 | Helped |
| Use current-latent amplitude and averaged guidance/current phase instead of averaging both amplitude and phase (ablation) | Pixelsmith · 2048² | FID: 63.173 -> 63.116 | Helped |
| Add the chess-like mask (ablation) | Pixelsmith · 2048² | FID: 64.473 -> 63.116 | Helped |
| Add overlapping-patch averaging (ablation) | Pixelsmith · 2048² | FID: 63.493 -> 63.116 | Helped |
Surprising findings
- FID barely distinguishes variants in the ablation (differences of about 1 point), so the authors argue FID cannot detect high-resolution artifacts and qualitative assessment is necessary.
- A two-step (intermediate resolution) cascade can yield finer small-scale details than one-step direct generation, despite the paper's emphasis on one-step generation.
Limitations
- The Slider value must be tuned manually per image and resolution, with no automatic selection.
- Single-step generation at very high scale requires high Slider values that suppress detail; cascading helps but adds time.
- Evaluation relies on FID/KID/IS/CLIP on 1,000 LAION prompts, which the authors note are inadequate for detecting high-resolution artifacts.
Experimental setup
Datasets: LAION-5B
Benchmarks: LAION-5B
Metrics: FID KID IS CLIP Score inference time
Baselines: SDXL ScaleCrafter DemoFusion HiDiffusion FouriScale AccDiffusion
Builds on: DiffInfinite SDXL DemoFusion MultiDiffusion
Closest papers by meaning
- Any-Size-Diffusion: Toward Efficient Text-Driven Synthesis for Any-Size HD Images (AAAI 2024 · similarity 0.87)
- DiffuseHigh: Training-Free Progressive High-Resolution Image Synthesis Through Structure Guidance (AAAI 2025 · similarity 0.87)
- DistriFusion: Distributed Parallel Inference for High-Resolution Diffusion Models (CVPR 2024 · similarity 0.87)
- ScaleCrafter: Tuning-free Higher-Resolution Visual Generation with Diffusion Models (ICLR 2024 · similarity 0.86)
- ResMaster: Mastering High-Resolution Image Generation via Structural and Fine-Grained Guidance (AAAI 2025 · similarity 0.86)
- Hierarchical Patch Diffusion Models for High-Resolution Video Generation (CVPR 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.