Research map / Image generation / Text-to-image diffusion
SnapGen: Taming High-Resolution Text-to-Image Models for Mobile Devices with Efficient Architectures and Training
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds an efficient 379M-parameter UNet T2I model with a tiny decoder, multi-level cross-architecture distillation from SD3.5-Large, and adversarial step distillation, enabling 1024px generation on phones in about 1.4 s.
Problem
Large T2I diffusion models are too big and slow for mobile devices, and prior efficient models are limited to low resolution (512px); the question is how to train a compact model from scratch for high-resolution on-device generation.
Approach
Systematically redesign the SDXL UNet (remove high-res self-attention, separable convs with expansion, trimmed FFN, MQA, early cross-attention, QK RMSNorm and 2D RoPE) and a tiny decoder, then train from scratch with flow matching using timestep-aware multi-level distillation from a DiT teacher and adversarial distribution-matching step distillation.
Key results
- SnapGen with 379M parameters reaches GenEval 0.66 and DPG-Bench 0.66, outperforming SDXL (2.6B, 0.55/0.59 reported) and IF-XL (5.5B, 0.61).
- On ImageNet-1K 256px, the efficient UNet (372M) achieves FID 2.06, matching SiT-XL (675M) with about one-third the FLOPs (38G vs 119G).
- The tiny decoder is about 36x smaller (1.38M params) than SDXL/SD3 decoders and generates 10242 px images on mobile in about 1.4 s total, with a 119 ms decode and 274 ms per UNet step on iPhone 16 Pro Max.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| remove self-attention from high-resolution stages (ablation) | SnapGen UNet · ImageNet-1K 256×256 | FID: 3.76 -> 3.12 | Helped |
| replace Conv with separable convolution (ablation) | SnapGen UNet · ImageNet-1K 256×256 | FID: 3.12 -> 3.38 | Hurt |
| expand intermediate channels in separable convolution (expansion ratio 2) (ablation) | SnapGen UNet · ImageNet-1K 256×256 | FID | Helped |
| reduce FFN expansion ratio to 3 (ablation) | SnapGen UNet · ImageNet-1K 256×256 | FID | No clear effect |
| replace MHSA with MQA (ablation) | SnapGen UNet · ImageNet-1K 256×256 | FID | No clear effect |
| inject conditional embeddings from the first stage (ablation) | SnapGen UNet · ImageNet-1K 256×256 | FID | Helped |
| add QK RMSNorm and RoPE positional embeddings (ablation) | SnapGen UNet · ImageNet-1K 256×256 | FID | Helped |
| use the tiny decoder instead of the SD3 decoder (ablation) | SnapGen decoder · COCO 2017 validation set, 1024×1024 decoding | PSNR: 27.92 -> 27.85 | No clear effect |
| add knowledge distillation from SD3.5-Large (ablation) | SnapGen · GenEval | score: 0.61 -> 0.66 | Helped |
| add knowledge distillation from SD3.5-Large (ablation) | SnapGen · DPG-Bench | score: 76.3 -> 81.1 | Helped |
| apply adversarial step distillation (ablation) | SnapGen · GenEval | score: 0.61 -> 0.66 | Helped |
Surprising findings
- Removing self-attention from high-resolution stages improved FID (3.76 to 3.12) while cutting FLOPs and latency.
- The 379M model without distillation already rivals billion-parameter models, and KD adds about five points on DPG-Bench and GenEval.
Limitations
- Few-step generation shows slight qualitative degradation relative to the 28-step baseline.
- Image-text alignment lags the SD3.5-Large teacher in the user study.
Experimental setup
Datasets: ImageNet-1K COCO 2017 Parti prompts PixArt prompts
Benchmarks: ImageNet-1K GenEval DPG-Bench MS-COCO CLIP score ImageReward
Metrics: FID GenEval DPG-Bench CLIP score ImageReward PSNR latency FLOPs parameters
Baselines: SDXL PixArt-α PixArt-Σ Sana Playgroundv2.5 IF-XL LUMINA-Next SD3 SD3.5-Large
Builds on: SDXL SD3 LADD DMD2 MobileDiffusion
Closest papers by meaning
- KOALA: Empirical Lessons Toward Memory-Efficient and Fast Diffusion Models for Text-to-Image Synthesis (NeurIPS 2024 · similarity 0.90)
- PixArt-$\alpha$: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis (ICLR 2024 · similarity 0.87)
- Würstchen: An Efficient Architecture for Large-Scale Text-to-Image Diffusion Models (ICLR 2024 · similarity 0.87)
- SnapGen-V: Generating a Five-Second Video within Five Seconds on a Mobile Device (CVPR 2025 · similarity 0.87)
- Upcycling Text-to-Image Diffusion Models for Multi-Task Capabilities (ICML 2025 · similarity 0.87)
- CtrLoRA: An Extensible and Efficient Framework for Controllable Image Generation (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.