Research map / Image generation / Text-to-image diffusion
MultiBooth: Towards Generating All Your Concepts in an Image from Text
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MultiBooth, a two-phase framework that learns each concept as a compact QFormer-derived embedding plus LoRA module, then composes multiple concepts in one image via bounding-box-restricted cross-attention for multi-concept customized text-to-image generation.
Problem
Multi-concept customized text-to-image generation suffers from low concept fidelity, feature confusion from joint training, and high inference cost when combining several user-provided concepts.
Approach
Learn each concept separately with a multi-modal QFormer encoder, adaptive concept normalization of the embedding norm, and a LoRA on key/value projections; then integrate concepts plug-and-play by restricting each concept's cross-attention to its bounding-box region, processing all concepts simultaneously rather than sequentially.
Key results
- Multi-concept setting: CLIP-I 0.714, Seg CLIP-I 0.713, CLIP-T 0.838, outperforming Cones2 (0.670/0.685/0.816) and Custom Diffusion.
- Training about 6 minutes per concept with per-concept module under 7MB, versus 10 min for DreamBooth at 3.3GB and 72MB for Custom Diffusion.
- Inference time rises only modestly with concepts: 8.29s for 2 concepts, 10.07s for 3, 10.53s for 4 on one RTX3090.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the MultiBooth single-concept learning and integration method | Stable Diffusion v1.5 · Single-concept | CLIP-I: 0.769 -> 0.783 | Helped |
| add the MultiBooth single-concept learning and integration method | Stable Diffusion v1.5 · Single-concept | Seg CLIP-I: 0.747 -> 0.761 | Helped |
| add the MultiBooth single-concept learning and integration method | Stable Diffusion v1.5 · Single-concept | CLIP-T: 0.813 -> 0.780 | Hurt |
| add the MultiBooth single-concept learning and integration method | Stable Diffusion v1.5 · Multi-concept | CLIP-I: 0.670 -> 0.714 | Helped |
| add the MultiBooth single-concept learning and integration method | Stable Diffusion v1.5 · Multi-concept | Seg CLIP-I: 0.685 -> 0.713 | Helped |
| add the MultiBooth single-concept learning and integration method | Stable Diffusion v1.5 · Multi-concept | CLIP-T: 0.828 -> 0.838 | Helped |
| add the Regional Customization Module (ablation) | MultiBooth · Multi-concept | CLIP-I: 0.691 -> 0.713 | Helped |
| add the Regional Customization Module (ablation) | MultiBooth · Multi-concept | Seg CLIP-I: 0.707 -> 0.712 | Helped |
| add the Regional Customization Module (ablation) | MultiBooth · Multi-concept | CLIP-T: 0.710 -> 0.838 | Helped |
| add the QFormer (ablation) | MultiBooth · Multi-concept | CLIP-I: 0.691 -> 0.713 | Helped |
| add the QFormer (ablation) | MultiBooth · Multi-concept | Seg CLIP-I: 0.694 -> 0.712 | Helped |
| add the QFormer (ablation) | MultiBooth · Multi-concept | CLIP-T: 0.823 -> 0.838 | Helped |
Surprising findings
- Removing the regional customization module makes the L2 norm of the customized embeddings grow to about 111-131, far above normal word embeddings, showing the embedding norm mismatch that ACN addresses.
Limitations
- Relies on user-specified or automatically generated bounding boxes for layout control.
- Fidelity is measured mainly by CLIP-based metrics; no human evaluation is reported in the main text.
- Built on Stable Diffusion v1.5 with experiments limited to twelve subjects.
Experimental setup
Datasets: DreamBooth dataset CustomConcept101
Benchmarks: DreamBooth dataset CustomConcept101
Metrics: CLIP-I Seg CLIP-I CLIP-T training time inference time
Baselines: Textual Inversion DreamBooth Custom Diffusion Cones2 LLM-grounded Diffusion BoxDiff
Builds on: Stable Diffusion BLIP-Diffusion Textual Inversion DreamBooth Custom Diffusion
Closest papers by meaning
- Concept Weaver: Enabling Multi-Concept Fusion in Text-to-Image Models (CVPR 2024 · similarity 0.89)
- MC^2: Multi-concept Guidance for Customized Multi-concept Generation (CVPR 2025 · similarity 0.89)
- Non-confusing Generation of Customized Concepts in Diffusion Models (ICML 2024 · similarity 0.87)
- LaTexBlend: Scaling Multi-concept Customized Generation with Latent Textual Blending (CVPR 2025 · similarity 0.87)
- Dis²Booth: Learning Image Distribution with Disentangled Features for Text-to-Image Diffusion Models (AAAI 2025 · similarity 0.87)
- LLM Blueprint: Enabling Text-to-Image Generation with Complex and Detailed Prompts (ICLR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.