Research map / Image generation / Text-to-image diffusion
SANA: Efficient High-Resolution Text-to-Image Synthesis with Linear Diffusion Transformers
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Sana, an efficient high-resolution (up to 4096px) text-to-image diffusion transformer combining a 32x deep-compression autoencoder, linear attention DiT without positional embeddings, a decoder-only LLM (Gemma-2) text encoder with complex human instruction, and a Flow-DPM-Solver.
Problem
Text-to-image diffusion models at high resolution (1K-4K) are computationally expensive and slow, and large models (e.g., FLUX-12B) are costly to train and run on consumer hardware.
Approach
Combine a 32x-compression autoencoder (AE-F32C32P1) with ReLU linear attention and Mix-FFN (depthwise conv + GLU) in a DiT without positional embeddings, use a decoder-only small LLM (Gemma-2) as text encoder with normalization and complex human instruction, and a rectified-flow adapted DPM-Solver for fast sampling.
Key results
- Sana-0.6B (590M params) reaches 0.64 GenEval and 84.3 DPG-Bench at 512px, outperforming PixArt-Sigma (0.52, 79.5) with 5x higher throughput.
- At 1024px, Sana-0.6B achieves GenEval 0.64 and DPG 83.6 versus FLUX-dev's 0.67 and 84.0, with about 39.5x higher throughput on A100.
- At 4096px, Sana-0.6B latency is 9.6 s versus 1023 s for FLUX-dev (about 104x speedup); W8A8 quantization brings 1024px latency on a laptop GPU to 0.37 s.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace full attention with linear attention (ablation) | DiT blocks · 256 | FID: 18.7 -> 21.5 | Hurt |
| add MixFFN after linear attention (ablation) | DiT blocks · 256 | FID: 21.5 -> 18.9 | Helped |
| add kernel fusion (ablation) | linear-attention and MixFFN blocks · 256 | FID: 18.9 -> 18.8 | No clear effect |
| add kernel fusion (ablation) | linear-attention and MixFFN blocks · 1024×1024, A100 | latency (ms): 2425 -> 2139 | Helped |
| use AE-F32C32P1 rather than AE-F8C16P4 at the same token count (ablation) | Sana autoencoder and diffusion pipeline · 1024×1024 image generation | FID: F32C32P1<$<$F16C32P2<$F8C16P4 | Helped |
| increase AE channels from 16 to 32 (ablation) | AE-F32 · AE reconstruction and generation | reconstruction metrics | Helped |
| use Gemma-2B-IT rather than T5-XXL as text encoder (ablation) | Sana text encoder · MJHQ-30K | FID: 6.1 -> 5.9 | Helped |
| quantize Sana with W8A8 (ablation) | Sana on laptop GPU · 1024px image generation | latency (s): 0.88 -> 0.37 | Helped |
| quantize Sana with W8A8 (ablation) | Sana on laptop GPU · MJHQ-30K | CLIP-Score: 28.5 -> 28.3 | Hurt |
| use Flow-DPM-Solver instead of Flow-Euler (ablation) | Sana sampling · text-to-image generation | sampling steps for convergence: 28-50 -> 14-20 | Helped |
Surprising findings
- Removing positional embeddings entirely from the DiT caused no measured quality loss, with 3x3 depthwise convolution in Mix-FFN providing implicit position information.
- Directly plugging a decoder-only LLM (Gemma-2) into cross-attention caused training instability with NaN losses, which was fixed by RMSNorm plus a small learnable scale on text embeddings.
Limitations
- Cannot fully guarantee safety and controllability of generated content.
- Remains weak on complex cases such as text rendering and generating faces and hands.
- Speed comparisons are measured on A100 with batch-size and step settings that vary per table.
Experimental setup
Datasets: MJHQ-30K GenEval DPG-Bench ImageReward prompts
Benchmarks: MJHQ-30K GenEval DPG-Bench ImageReward
Metrics: FID CLIP score GenEval overall DPG-Bench overall ImageReward throughput latency
Baselines: PixArt-alpha PixArt-Sigma SDXL SD3-medium FLUX-dev FLUX-schnell Hunyuan-DiT DALLE3 LUMINA-Next PlayGroundv2.5
Builds on: DiT PixArt-Sigma DPM-Solver++ EfficientViT Rectified Flow
Closest papers by meaning
- SANA 1.5: Efficient Scaling of Training-Time and Inference-Time Compute in Linear Diffusion Transformer (ICML 2025 · similarity 0.91)
- Meissonic: Revitalizing Masked Generative Transformers for Efficient High-Resolution Text-to-Image Synthesis (ICLR 2025 · similarity 0.88)
- PixArt-$\alpha$: Fast Training of Diffusion Transformer for Photorealistic Text-to-Image Synthesis (ICLR 2024 · similarity 0.88)
- Text-Aware Real-World Image Super-Resolution via Diffusion Model with Joint Segmentation Decoders (NeurIPS 2025 · similarity 0.87)
- SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis (ICLR 2024 · similarity 0.87)
- Würstchen: An Efficient Architecture for Large-Scale Text-to-Image Diffusion Models (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.