Research map / Architectures and efficiency / Attention and transformers
TransGAN: Two Pure Transformers Can Make One Strong GAN, and That Can Scale Up
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TransGAN instead of StyleGAN-V2 + DiffAug. | CIFAR-10 image generation · CIFAR-10 | FID: 9.89 -> 9.26 | Helped |
| Use TransGAN instead of StyleGAN-V2 + DiffAug. | STL-10 image generation · STL-10 | IS: 10.31* -> 10.43 | Helped |
| Use TransGAN instead of StyleGAN-V2 + DiffAug. | STL-10 image generation · STL-10 | FID: 19.15* -> 18.28 | Helped |
| Use TransGAN instead of StyleGAN-V2 + DiffAug. | CelebA 128x128 image generation · CelebA 128x128 | FID: 5.40* -> 5.28 | Helped |
| Add DiffAug (ablation) | TransGAN · CIFAR-10 | FID: 22.53 -> 9.26 | Helped |
| Add DiffAug (ablation) | TransGAN · CIFAR-10 | IS: 8.36 -> 9.02 | Helped |
| Add DiffAug (ablation) | WGAN-GP · CIFAR-10 | IS: 6.49 -> 6.29 | Hurt |
| Add DiffAug (ablation) | AutoGAN · CIFAR-10 | FID: 12.42 -> 12.72 | Hurt |
| Replace grid self-attention with Nyström or axis self-attention (ablation) | TransGAN · CelebA 64x64, CelebA 128x128, and LSUN Church 256x256 | FID: Grid: 9.89, 10.58, 20.39; Nyström: 13.47, 17.42, 39.92; Axis: 12.39, 13.95, 29.30 | Hurt |
| Replace standard self-attention with grid self-attention (ablation) | TransGAN · CelebA 128x128 and LSUN Church 256x256 | FID: OOM -> 10.58; OOM -> 20.39 | Helped |
| Add a multi-scale discriminator (ablation) | TransGAN with grid self-attention · CelebA 64x64, CelebA 128x128, and LSUN Church 256x256 | FID: 9.89 -> 9.28; 10.58 -> 8.03; 20.39 -> 15.29 | Helped |
| Add modified normalization (ablation) | TransGAN with multi-scale discriminator · CelebA 64x64, CelebA 128x128, and LSUN Church 256x256 | FID: 9.28 -> 7.05; 8.03 -> 7.13; 15.29 -> 13.27 | Helped |
Closest papers by meaning
- Generative Adversarial Transformers (ICML 2021 · similarity 0.90)
- Improved Transformer for High-Resolution GANs (NeurIPS 2021 · similarity 0.89)
- ViTGAN: Training GANs with Vision Transformers (ICLR 2022 · similarity 0.89)
- The GAN is dead; long live the GAN! A Modern GAN Baseline (NeurIPS 2024 · similarity 0.88)
- MCGAN: Enhancing GAN Training with Regression-Based Generator Loss (AAAI 2025 · similarity 0.88)
- E$^2$GAN: Efficient Training of Efficient GANs for Image-to-Image Translation (ICML 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.