Research map / Image generation / GANs
SAN: Inducing Metrizability of GAN with Discriminative Normalized Linear Layer
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Derives metrizable conditions (direction optimality, separability, injectivity) under which a GAN discriminator induces a distance via sliced optimal transport, and proposes SAN, a modified GAN training scheme that puts a normalized final linear layer on a hypersphere and adds a direction-optimizing loss.
Problem
GAN minimax optimization is not known to provide generator gradients that reduce a true distance between the generator and target distributions, since the optimal-discriminator assumption rarely holds in practice.
Approach
Connects GAN discriminators in inner-product form to augmented sliced Wasserstein distances, deriving sufficient metrizable conditions; SAN enforces direction optimality by normalizing the last linear layer and adding a separate objective that updates the direction with stop-gradient features, requiring only small code changes to existing GANs.
Key results
- StyleSAN-XL reaches FID 1.36 on CIFAR10 versus 1.85 for StyleGAN-XL and FID 2.14 versus 2.30 on ImageNet 256x256, with IS 274.20 versus 265.12.
- On DCGAN, SAN lowers FID on CIFAR10 across hinge, saturating, and non-saturating losses (e.g., hinge 24.07 to 20.23).
- With BigGAN on CIFAR10, hinge SAN achieves FID 6.20 versus 8.25 for the reproduced hinge GAN.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply SAN training with hinge, saturating, or non-saturating loss | DCGAN · CIFAR10 | FID: 24.07{ $\pm$0.56} -> \textbf{20.23}{ $\pm$0.86}; 25.63{ $\pm$0.98} -> \textbf{20.62}{ $\pm$0.94}; 24.90{ $\pm$0.21} -> \ | Helped |
| Apply SAN training with hinge, saturating, or non-saturating loss | DCGAN · CelebA (128×128) | FID: 32.51{ $\pm$2.53} -> \textbf{27.79}{ $\pm$1.60}; 37.33{ $\pm$1.02} -> \textbf{28.16}{ $\pm$1.60}; 28.22{ $\pm$2.16} -> \ | Helped |
| Apply SAN training with hinge loss | BigGAN · CIFAR10 | FID: 8.25{ $\pm$0.82} -> \textbf{6.20}{ $\pm$0.27} | Helped |
| Apply SAN training with hinge loss | BigGAN · CIFAR10 | IS: 9.05{ $\pm$0.05} -> \textbf{9.16}{ $\pm$0.08} | Helped |
| Apply SAN training with hinge loss | BigGAN · CIFAR100 | FID: 10.73{ $\pm$0.16} -> \textbf{8.05}{ $\pm$0.04} | Helped |
| Apply SAN training with hinge loss | BigGAN · CIFAR100 | IS: 10.56{ $\pm$0.01} -> \textbf{10.72}{ $\pm$0.05} | Helped |
| Apply SAN training to StyleGAN-XL | StyleGAN-XL · CIFAR10 | FID: (1.85) -> \textbf{1.36} | Helped |
| Apply SAN training to StyleGAN-XL | StyleGAN-XL · ImageNet (256×256) | FID: 2.30 -> \textbf{2.14} | Helped |
| Apply SAN training to StyleGAN-XL | StyleGAN-XL · ImageNet (256×256) | IS: 265.12 -> \textbf{274.20} | Helped |
Surprising findings
- Wasserstein GAN discriminators tend not to satisfy separability, offering an alternative explanation for its training instability.
Limitations
- Direction optimality is only approximated via a stop-gradient auxiliary objective, and for general GANs the normalization terms are approximated per batch.
- Injectivity cannot be directly controlled by loss design and is not guaranteed by the training scheme.
Experimental setup
Datasets: CIFAR10 CelebA (128x128) CIFAR100 ImageNet (256x256) synthetic 8-mode mixture of Gaussians
Benchmarks: CIFAR10 CelebA CIFAR100 ImageNet 256x256 mixture of Gaussians
Metrics: FID IS
Baselines: DCGAN with hinge/saturating/non-saturating loss BigGAN StyleGAN-XL Wasserstein GAN
Builds on: Wasserstein GAN augmented sliced Wasserstein distance BigGAN StyleGAN-XL spectral normalization
Closest papers by meaning
- Training Generative Adversarial Networks with Limited Data (NeurIPS 2020 · similarity 0.90)
- SGAN: An Alternative Training of Generative Adversarial Networks (CVPR 2018 · similarity 0.89)
- GLeaD: Improving GANs With a Generator-Leading Task (CVPR 2023 · similarity 0.89)
- TRAINING GENERATIVE ADVERSARIAL NETWORKS VIA PRIMAL-DUAL SUBGRADIENT METHODS: A LAGRANGIAN PERSPECTIVE ON GAN (ICLR 2018 · similarity 0.89)
- Hidden Convexity of Wasserstein GANs: Interpretable Generative Models with Closed-Form Solutions (ICLR 2022 · similarity 0.89)
- MCGAN: Enhancing GAN Training with Regression-Based Generator Loss (AAAI 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.