Research map / Image generation / Autoregressive and latent generation
Native-Resolution Image Synthesis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes NiT, a diffusion transformer that trains on images at native resolutions and aspect ratios via packed variable-length tokens, achieving SOTA ImageNet 256/512 FID with one model and zero-shot generalization to unseen resolutions and aspect ratios.
Problem
Standard diffusion transformers force images to fixed square resolutions, discarding native resolution and aspect ratio information and generalizing poorly to unseen resolutions and aspect ratios.
Approach
Pack variable-length latent token sequences from images of arbitrary resolution into one sequence, use FlashAttention varlen with cumulative sequence lengths for per-image full attention, axial 2D RoPE for positions, and packed AdaLN for per-instance conditioning, so a single model is trained at native resolution.
Key results
- NiT-XL achieves FID 2.03 at 256x256 and 1.45 at 512x512 with a single model, the best 512 FID reported (outperforming EDM2-XXL at 1.81).
- NiT-XL attains FID 4.52 at unseen 1024x1024 and 6.51 at 1536x1536 without training at those resolutions, while EDM2-L and FlowDCN fail beyond 1024.
- On aspect ratios, NiT-XL reaches FID 4.11 at 9:16 and 5.27 at 16:9, while EDM2-L and SiT-REPA degrade sharply at extreme ratios such as 1:3.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use NiT-XL with one native-resolution model instead of resolution-specific models | ImageNet class-guided generation · ImageNet 512×512 | FID: 1.81 -> 1.45 | Helped |
| Use NiT-XL with one native-resolution model instead of resolution-specific models | ImageNet class-guided generation · ImageNet 256×256 | FID: 1.42 -> 2.03 | Hurt |
| Use NiT-XL rather than the strongest listed baseline | ImageNet class-guided generation · ImageNet 768×768 | FID: 9.02 -> 4.05 | Helped |
| Use NiT-XL rather than the strongest listed baseline | ImageNet class-guided generation · ImageNet 1024×1024 | FID: 18.64 -> 4.52 | Helped |
| Use NiT-XL rather than the strongest listed baseline | ImageNet class-guided generation · ImageNet 1536×1536 | FID: 41.170 -> 6.51 | Helped |
| Use NiT-XL rather than the strongest listed baseline | ImageNet class-guided generation · ImageNet 2048×2048 | FID: 69.88 -> 24.76 | Helped |
| Add 256×256 and 512×512 training images to native-resolution training (ablation) | NiT-B · ImageNet 256×256 | FID: 23.22 -> 15.46 | Helped |
| Add 256×256 and 512×512 training images to native-resolution training (ablation) | NiT-B · ImageNet 512×512 | FID: 15.75 -> 9.56 | Helped |
| Add 256×256 and 512×512 training images to native-resolution training (ablation) | NiT-B · ImageNet 768×768 | FID: 16.77 -> 12.42 | Helped |
| Remove native-resolution images and train only on 256×256 and 512×512 images (ablation) | NiT-B · ImageNet 768×768 | FID: 12.42 -> 33.50 | Hurt |
| Increase training token budget from 131B to 197B | NiT-XL · ImageNet 256×256 | FID: 2.16 -> 2.03 | Helped |
| Increase training token budget from 131B to 197B | NiT-XL · ImageNet 512×512 | FID: 1.57 -> 1.45 | Helped |
Surprising findings
- Training solely on ImageNet native-resolution data yields zero-shot generalization to unseen resolutions up to 1536x1536 and extreme aspect ratios, despite no such training examples.
- Models trained only on fixed 256 and 512 resolutions generalize poorly to 768x768 compared with mixtures including native-resolution data.
Limitations
- Generalization to extremely high resolutions and extreme aspect ratios is still unsatisfactory (e.g., FID 24.76 at 2048x2048).
- Requires substantial compute for comprehensive multi-scale training.
- Evaluated mainly on ImageNet class-conditional generation; generality to other data domains such as video is untested.
Experimental setup
Datasets: ImageNet SAM COCO-2014
Benchmarks: ImageNet 256x256 ImageNet 512x512 COCO-2014
Metrics: FID sFID IS Precision Recall CLIP score
Baselines: DiT-XL/2 SiT-XL/2 SiT-REPA FlowDCN FiTv2-XL EDM2-L EDM2-XXL VAR-2B LlamaGen-3B MaskGiT SD-v1.5 SDXL-Turbo
Builds on: DiT SiT FlashAttention-2 EDM2 REPA
Closest papers by meaning
- DiffuseHigh: Training-Free Progressive High-Resolution Image Synthesis Through Structure Guidance (AAAI 2025 · similarity 0.90)
- ScaleCrafter: Tuning-free Higher-Resolution Visual Generation with Diffusion Models (ICLR 2024 · similarity 0.89)
- Scalable High-Resolution Pixel-Space Image Synthesis with Hourglass Diffusion Transformers (ICML 2024 · similarity 0.88)
- Reconstruction vs. Generation: Taming Optimization Dilemma in Latent Diffusion Models (CVPR 2025 · similarity 0.88)
- ResMaster: Mastering High-Resolution Image Generation via Structural and Fine-Grained Guidance (AAAI 2025 · similarity 0.88)
- SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis (ICLR 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.