Research map / Image generation / Editing and style transfer

WeditGAN: Few-Shot Image Generation via Latent Space Relocation

AAAI 2024 · Image generation › Editing and style transfer · approach: Image-to-image translation · 13 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Transfers a pretrained StyleGAN2 to few-shot target domains by learning a single constant latent offset Δw in W+ while freezing mapping and synthesis networks, with variants adding perpendicular regularization, per-sample intensity modules, and contrastive losses.

Problem

Few-shot image generation with GANs overfits when trained on only a handful (~10) of target images; model transfer from large source domains is often limited by the trade-off between source diversity and target fidelity.

Approach

Relocate the source W+ latent space to the target domain via one learned constant offset Δw per layer, keeping the pretrained generator intact so the source-target mapping is one-to-one and overfitting is avoided; only 512×n parameters are trained.

Key results

• On FFHQ to Sketches, WeditGAN achieves FID 35.41 versus 40.52 for the best prior baseline (RICK); the perp variant reaches LPIPS 0.4504 and the alpha variant FID 34.13.
• On Sunglasses, the alpha variant reaches FID 17.06 against 25.09 for RICK, and WeditGAN alone reaches LPIPS 0.6254 versus 0.5992 for RICK.
• WeditGAN trains only 10,240 generator parameters (about 0.04% of the generator), taking 46 minutes on a single TITAN RTX versus 1h 5min to 4h 51min for baselines.

Measured effects reported in the paper

Surprising findings

• A unified constant Δw shared across all layers performs much worse (FID 88.73 on Sketches), and adaptive per-z Δw overfits on 10 images.
• Finetuning the full discriminator (no FreezeD) did not cause diversity degradation; FreezeD variants gave worse FID on Sketches.

Limitations

• Assumes source and target domains share the same object category and manifold shape; fails for e.g. FFHQ to Haunted houses.
• A single constant Δw cannot cover target datasets composed of multiple distinct styles.
• Relies on the same pretrained source model; mixed-domain targets lead to fused characteristics.

Experimental setup

Datasets: FFHQ LSUN Church LSUN Car Sketches Babies Sunglasses Amedeo Modigliani paintings Raphael paintings Otto Dix paintings Haunted houses Wrecked cars

Benchmarks: FFHQ to Sketches FFHQ to Babies FFHQ to Sunglasses FFHQ to Amedeo FFHQ to Raphael FFHQ to Otto LSUN Church to Haunted LSUN Car to Wrecked

Metrics: FID@5k KID@5k intra-cluster LPIPS@1k

Baselines: CDC DCL RSSA DWSC AdAM RICK

Builds on: StyleGAN2-ADA DCL FreezeD StyleCLIP

Closest papers by meaning

• Delving StyleGAN Inversion for Image Editing: A Foundation Latent Space Viewpoint (CVPR 2023 · similarity 0.89)
• Exploiting Spatial Dimensions of Latent in GAN for Real-Time Image Editing (CVPR 2021 · similarity 0.88)
• Using latent space regression to analyze and leverage compositionality in GANs (ICLR 2021 · similarity 0.88)
• Navigating the GAN Parameter Space for Semantic Image Editing (CVPR 2021 · similarity 0.88)
• Analyzing and Improving the Image Quality of StyleGAN (CVPR 2020 · similarity 0.88)
• Few-shot Cross-domain Image Generation via Inference-time Latent-code Learning (ICLR 2023 · similarity 0.88)