Research map / Label-efficient and robust learning / Domain adaptation and generalization
Few-shot Hybrid Domain Adaptation of Image Generator
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Few-shot Hybrid Domain Adaptation (HDA), adapting a StyleGAN2 generator to the hybrid of several target domains with a discriminator-free directional subspace loss over pre-trained encoder (Swin, DINOv2) embedding subspaces.
Problem
Adapting a pre-trained image generator to a hybrid of multiple target domains (integrating attributes of several few-shot domains) when no real hybrid-domain images are available, and discriminator-based DA overfits to easy domains.
Approach
Replace the discriminator with a frozen pre-trained image encoder: target-domain reference images define embedding subspaces; a distance loss pulls generated embeddings toward all target subspaces while a direction loss keeps the adaptation perpendicular to the subspaces from the source embedding, preserving source characteristics.
Key results
- On 10-shot HDA smile-sunglass, the method reports CLIP-Score 25.65 vs 20.23 for the best baseline (NADA).
- On 10-shot DA, the method achieves CLIP-Score 24.13 on sunglasses and 22.83 on sketch, exceeding all listed baselines.
- Adaptation takes about 3 minutes (300 iterations, batch size 4) on a single TITAN GPU, versus 240 minutes for sequential and DoRM-style methods on two domains.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use directional subspace loss (distance and direction losses) instead of the strongest reported baselines | StyleGAN2 HDA · 10-shot smile-sunglass, baby-smile, and baby-sketch | CLIP-Score: 20.23 -> 25.65; 20.45 -> 23.08; 22.69 -> 24.91 | Helped |
| Use directional subspace loss (distance and direction losses) instead of the strongest reported baselines | StyleGAN2 HDA · 10-shot smile-sunglass and baby-smile | IS: 1.94 -> 2.4; 2.12 -> 2.94 | Helped |
| Use directional subspace loss (distance and direction losses) instead of the strongest reported baselines | StyleGAN2 HDA · 10-shot baby-sketch | IS: +0.28 | Helped |
| Add the direction loss to distance loss (ablation) | StyleGAN2 DA · 10-shot sunglasses | IS: 1.91 -> 2.67 | Helped |
| Add the direction loss to distance loss (ablation) | StyleGAN2 DA · 10-shot sunglasses | ID: 0.249 -> 0.414 | Helped |
| Add the direction loss to distance loss (ablation) | StyleGAN2 DA · 10-shot baby | IS: 2.56 -> 3.71 | Helped |
| Add the direction loss to distance loss (ablation) | StyleGAN2 DA · 10-shot baby | ID: 0.178 -> 0.353 | Helped |
| Use direction loss alone rather than distance loss alone (ablation) | StyleGAN2 DA · 10-shot sunglasses | IS: 1.91 -> 2.61 | Helped |
| Use direction loss alone rather than distance loss alone (ablation) | StyleGAN2 DA · 10-shot baby | IS: 2.56 -> 3.5 | Helped |
| Use 1-shot instead of 5-shot or 10-shot references | StyleGAN2 HDA · Hybrid domains | cross-domain consistency | Hurt |
Surprising findings
- Using only the distance loss can cause model collapse, with distinct generated images projecting to the same subspace point; the direction loss is needed for cross-domain consistency.
- With only one reference image per domain, the subspace degenerates to a point, leading to lower cross-domain consistency than 5 or 10 shots.
Limitations
- Domain loss weights do not necessarily match the proportion of visual characteristics and require empirical tuning.
- Performance degrades with 1-shot references due to subspace degeneracy.
- Evaluated mainly on FFHQ face domains with CLIP-based metrics that may carry bias.
Experimental setup
Datasets: FFHQ ImageNet-22k
Benchmarks: FFHQ-based HDA targets (smile-sunglass, baby-smile, baby-sketch) FFHQ-based DA targets (baby, smile, sunglasses, sketch)
Metrics: CLIP-Score Inception Score identity similarity Intra-LPIPS
Baselines: Sequential adaptation DoRM NADA CDC RSSA AdAM
Builds on: StyleGAN2 NADA CDC DoRM RSSA
Closest papers by meaning
- Towards Diverse and Faithful One-shot Adaption of Generative Adversarial Networks (NeurIPS 2022 · similarity 0.88)
- DomainGallery: Few-shot Domain-driven Image Generation by Attribute-centric Finetuning (NeurIPS 2024 · similarity 0.88)
- Invertible Projection and Conditional Alignment for Multi-Source Blended-Target Domain Adaptation (AAAI 2025 · similarity 0.87)
- Domain-Rectifying Adapter for Cross-Domain Few-Shot Segmentation (CVPR 2024 · similarity 0.86)
- Mind the Gap Between Prototypes and Images in Cross-domain Finetuning (NeurIPS 2024 · similarity 0.86)
- Unified Language-driven Zero-shot Domain Adaptation (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.