Research map / Image generation / Text-to-image diffusion
DomainGallery: Few-shot Domain-driven Image Generation by Attribute-centric Finetuning
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DomainGallery, a LoRA-based DreamBooth-style finetuning of Stable Diffusion v1.4 for few-shot domain-driven image generation, using prior attribute erasure, attribute disentanglement, transfer-based similarity regularization, and inference-time attribute enhancement.
Problem
Few-shot domain-driven image generation: adapting text-to-image models to a target domain from about 10 images so that the domain attributes can be combined with arbitrary categories and extra text attributes, which prior methods fail to handle (leaked prior attributes, attribute entanglement, overf
Approach
Treat a domain as the set of attributes common to its images, bind non-categorical domain attributes to an identifier token, and add four attribute-centric techniques: pre-finetuning erasure of the identifier's prior attributes, a disentanglement loss keeping the category word's outputs unchanged, a transfer-based similarity consistency loss computed on paired denoised latents (no decoding), and C
Key results
- On intra-category generation, DomainGallery reaches FID 44.86 on CUFS sketches versus 52.80 for DreamBooth+LoRA and 51.73 for DomainStudio, with higher I-LPIPS (0.5060).
- On Van Gogh houses, DomainGallery achieves KID 32.20 and I-LPIPS 0.7255, the best among the compared methods.
- DomainGallery does not win everywhere: DreamBooth without LoRA has better KID on watercolor dogs and wrecked cars (its overfitting gives low KID), and DreamBooth+LoRA has the best FID on FFHQ sunglasses (41.22 vs 43.10).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use DomainGallery instead of the baselines | Few-shot intra-category generation · CUFS sketches, FFHQ sunglasses, Van Gogh houses, watercolor dogs, wrecked cars | FID/KID and I-LPIPS | Mixed |
| Remove prior attribute erasure before finetuning (ablation) | DomainGallery · Cross-category generation on CUFS sketches | prior-attribute leakage | Hurt |
| Remove domain-category disentanglement loss (ablation) | DomainGallery · Cross-category generation on CUFS sketches | domain/category attribute leakage | Hurt |
| Increase the similarity-consistency loss weight (ablation) | DomainGallery · CUFS sketches | I-LPIPS | Helped |
| Increase the similarity-consistency loss weight above 1.0 (ablation) | DomainGallery · CUFS sketches | FID | Hurt |
| Apply attribute enhancement to DreamBooth, DreamBooth+LoRA, or DomainStudio (ablation) | Baseline methods · Cross-category generation | fidelity | No clear effect |
| Use V-uncond rather than VN-N enhancement (ablation) | DomainGallery · Cross-category generation | domain-attribute strength | Helped |
Surprising findings
- Enhancing the identifier token alone at inference does not improve fidelity for baselines; the attribute enhancement only helps once domain attributes are properly learned into the identifier.
- Standard DreamBooth's low KID scores on some datasets stem from severe overfitting rather than good quality.
Limitations
- Handles only a single category word per dataset, so multi-category datasets need per-image category words (not tested).
- Performance on content-based domains (e.g., FFHQ sunglasses) is weaker, and some cross-category samples were cherry-picked.
- Composite domains with subtle shared attributes may not be well defined.
Experimental setup
Datasets: CUFS sketches FFHQ sunglasses Van Gogh houses watercolor dogs wrecked cars
Benchmarks: CUFS sketches FFHQ sunglasses Van Gogh houses watercolor dogs wrecked cars
Metrics: FID KID intra-cluster LPIPS
Baselines: DreamBooth DreamBooth+LoRA DomainStudio
Builds on: DreamBooth DomainStudio LoRA Stable Diffusion DDIM
Closest papers by meaning
- Few-shot Hybrid Domain Adaptation of Image Generator (ICLR 2024 · similarity 0.88)
- JeDi: Joint-Image Diffusion Models for Finetuning-Free Personalized Text-to-Image Generation (CVPR 2024 · similarity 0.85)
- Localizing and Editing Knowledge In Text-to-Image Generative Models (ICLR 2024 · similarity 0.85)
- Domain-RAG: Retrieval-Guided Compositional Image Generation for Cross-Domain Few-Shot Object Detection (NeurIPS 2025 · similarity 0.85)
- Image is All You Need to Empower Large-scale Diffusion Models for In-Domain Generation (CVPR 2025 · similarity 0.85)
- Diffusion Self-Distillation for Zero-Shot Customized Image Generation (CVPR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.