Research map / Image generation / Text-to-image diffusion
Dis²Booth: Learning Image Distribution with Disentangled Features for Text-to-Image Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Dis2Booth, which personalizes Stable Diffusion XL to generate varied multi-instance image distributions by splitting LoRA into an Anchor LoRA (common features) and a Delta LoRA (variations), trained with unsupervised contextual and delta losses plus asynchronous optimization.
Problem
Existing personalization methods learn a single instance or concept and fail on varied data (multiple individuals sharing common traits), causing category drift, overfitting and fidelity loss.
Approach
Decompose the varied data distribution into shared common features and per-sample variations, model them with two LoRA modules constrained unsupervised via momentum-queue contextual loss (relaxed EMD on cosine distances) and a thresholded delta loss, and update them asynchronously.
Key results
- On Animal Faces, Dis2Booth achieves FID 27.63 versus 32.50 for DreamBooth and 52.30 for WaveGAN (lower is better).
- On Flowers, Dis2Booth achieves FID 58.15 versus 65.10 for HAE and 94.91 for Textual Inversion.
- Dis2Booth reports LPIPS 0.5356 on Animal Faces and 0.5189 on Flowers, comparable to the diversity of other personalization baselines.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Dis2Booth instead of DreamBooth | personalized image generation · Animal Faces | FID: 32.50 -> 27.63 | Helped |
| Use Dis2Booth instead of DreamDistribution | personalized image generation · Animal Faces | LPIPS: 0.5351 -> 0.5356 | Helped |
| Use Dis2Booth instead of HAE | personalized image generation · Flowers | FID: 65.10 -> 58.15 | Helped |
| Use Dis2Booth instead of DreamDistribution | personalized image generation · Flowers | LPIPS: 0.5207 -> 0.5189 | Hurt |
Surprising findings
- Textual inversion-based methods get artificially high LPIPS because they mix in features from other categories, so LPIPS alone overstates diversity.
Limitations
- Can crash or degrade when fine-tuning high-resolution pretrained diffusion models with low-resolution images.
- Overfits when there are very few reference images and training runs too long.
- Evaluated only on Animal Faces and Flowers with FID/LPIPS; no human evaluation.
Experimental setup
Datasets: Animal Faces Flowers
Benchmarks: Animal Faces Flowers
Metrics: FID LPIPS
Baselines: WaveGAN HAE Textual Inversion DreamDistribution DreamBooth
Builds on: DreamBooth LoRA Stable Diffusion XL ZipLoRA MVDream
Closest papers by meaning
- DreamDistribution: Learning Prompt Distribution for Diverse In-distribution Generation (ICLR 2025 · similarity 0.89)
- JeDi: Joint-Image Diffusion Models for Finetuning-Free Personalized Text-to-Image Generation (CVPR 2024 · similarity 0.89)
- PaRa: Personalizing Text-to-Image Diffusion via Parameter Rank Reduction (ICLR 2025 · similarity 0.88)
- Mining your own secrets: Diffusion Classifier Scores for Continual Personalization of Text-to-Image Diffusion Models (ICLR 2025 · similarity 0.87)
- DisenBooth: Identity-Preserving Disentangled Tuning for Subject-Driven Text-to-Image Generation (ICLR 2024 · similarity 0.87)
- MultiBooth: Towards Generating All Your Concepts in an Image from Text (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.