Research map / Image generation / GANs
Spectrum Translation for Refinement of Image Generation (STIG) Based on Contrastive Learning and Spectral Filter Profile
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes STIG, a contrastive-learning-based image-to-image spectrum translation framework that refines the magnitude spectrum of GAN and diffusion-generated images to reduce frequency discrepancies and improve FID, while making frequency-based fake detectors fail.
Problem
Generated images from GANs and diffusion models show spectral discrepancies (checkerboard aliasing from upsampling and missing high-frequency components), which hurt image quality and make them easy to detect with frequency-based detectors.
Approach
Treat the magnitude spectrum of generated images as an input domain and learn an unpaired translation to the real spectrum using adversarial, patch-wise contrastive (NCE), SSIM reconstruction, a chessboard-integration spectral discriminator, and a low-frequency regularizer; analyze diffusion denoising via a Wiener filter to motivate the high-frequency deficit.
Key results
- STIG reduces magnitude-spectrum FID on GANs to 6.97 (CycleGAN, vs 58.75 original) and 6.30 (StarGAN, vs 192.48), outperforming SpectralGAN and FFL.
- On diffusion models STIG lowers image FID by 26.1% on average (e.g., DDIM-Face 36.70 to 22.19) and spectral FID by about 67%.
- Frequency-based CNN and ViT detectors that reach about 99% accuracy on original fakes drop to about 57% and 54% average accuracy on STIG-refined images.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Apply STIG spectrum translation | CycleGAN outputs · CycleGAN | spectrum FID: 58.75 -> 6.97 | Helped |
| Apply STIG spectrum translation | StarGAN outputs · StarGAN | spectrum FID: 192.48 -> 6.30 | Helped |
| Apply STIG spectrum translation | StarGAN2 outputs · StarGAN2 | spectrum FID: 15.03 -> 3.10 | Helped |
| Apply STIG spectrum translation | StyleGAN outputs · StyleGAN | spectrum FID: 3.59 -> 2.37 | Helped |
| Apply STIG spectrum translation | DDPM outputs · DDPM-Face | spectrum FID: 43.54 -> 21.91 | Helped |
| Apply STIG spectrum translation | DDPM outputs · DDPM-Church | spectrum FID: 59.66 -> 22.53 | Helped |
| Apply STIG spectrum translation | DDIM outputs · DDIM-Face | spectrum FID: 51.49 -> 8.56 | Helped |
| Apply STIG spectrum translation | DDIM outputs · DDIM-Church | spectrum FID: 55.25 -> 16.47 | Helped |
| Add low-frequency loss L_lf (ablation) | STIG · CycleGAN | image FID: 60.62 -> 59.85 | Helped |
| Add spectral discriminator loss L_spec (ablation) | STIG without L_lf · CycleGAN | image FID: 60.70 -> 60.62 | No clear effect |
| Add low-frequency loss L_lf (ablation) | STIG · StarGAN2 | image FID: 13.59 -> 13.36 | Helped |
| Add spectral discriminator loss L_spec (ablation) | STIG without L_lf · StarGAN2 | image FID: 13.58 -> 13.59 | No clear effect |
Surprising findings
- Spatial-domain spectral regularizers (Durall et al., Jung and Keuper) sometimes increase spectral discrepancy, e.g., for StarGAN2 and StyleGAN.
- SpectralGAN breaks the color tone of generated images, which the auxiliary low-frequency loss helps preserve.
Limitations
- Evaluation limited to low resolution (128x128 for DMs, 256x256 for GANs) and a small set of architectures and datasets.
- Detectors are trained and tested only on the paper's own generated datasets with a fixed training recipe.
Experimental setup
Datasets: CycleGAN horse2zebra/apple2orange/summer2winter CelebA AFHQ FFHQ LSUN Church
Benchmarks: CycleGAN StarGAN StarGAN2 StyleGAN DDPM-Face DDPM-Church DDIM-Face DDIM-Church
Metrics: FID log frequency distance (LFD) detection accuracy
Baselines: Durall et al. spectral regularization Jung and Keuper FFL SpectralGAN
Builds on: CycleGAN SpectralGAN focal frequency loss contrastive unpaired translation DDPM
Closest papers by meaning
- Spectral Distribution Aware Image Generation (AAAI 2021 · similarity 0.88)
- Fourier Spectrum Discrepancies in Deep Network Generated Images (NeurIPS 2020 · similarity 0.88)
- On the Frequency Bias of Generative Models (NeurIPS 2021 · similarity 0.87)
- SDEdit: Guided Image Synthesis and Editing with Stochastic Differential Equations (ICLR 2022 · similarity 0.87)
- You Only Sample Once: Taming One-Step Text-to-Image Synthesis by Self-Cooperative Diffusion GANs (ICLR 2025 · similarity 0.84)
- Not All Parameters Matter: Masking Diffusion Models for Enhancing Generation Ability (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.