Research map / Image generation / Text-to-image diffusion
LaTexBlend: Scaling Multi-concept Customized Generation with Latent Textual Blending
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LaTexBlend, which fine-tunes each concept into a compact latent textual representation (after text encoder and projection) stored in a concept bank and blends them at inference for scalable multi-concept customized text-to-image generation.
Problem
Multi-concept customized text-to-image generation degrades in quality (concept omission, blending, distortion, layout collapse) and efficiency as the number of concepts grows, due to denoising deviation induced by customization.
Approach
Represent each customized concept as compact latent textual features (K, V after the text encoder and linear projections) learned with an auxiliary frozen base textual flow that forces concept information into the concept tokens; blend these features directly in latent textual space at inference, plus a blending guidance based on cross-attention overlap.
Key results
- LaTexBlend reaches SI_CLIP 0.8052 and SDINO 0.6564 in single-concept ablation, with text alignment 0.3684, versus w/o base flow (0.5861 / 0.4337).
- In a user study with 25 participants, LaTexBlend scored 4.33 concept alignment, 4.16 prompt alignment, and 4.76 overall quality, the highest of all compared methods (best baseline MuDI 3.24 / 2.83 / 3.54).
- Fine-tuning cost grows linearly with concept count and adds no inference cost as concepts increase, unlike OMG whose inference cost rises with concepts.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add the frozen base embedding flow (ablation) | LaTexBlend · Customized generation | Textual alignment (ST_CLIP): 0.3718 -> 0.3684 | No clear effect |
| Add the frozen base embedding flow (ablation) | LaTexBlend · Customized generation | Concept alignment (SI_CLIP): 0.5861 -> 0.8052 | Helped |
| Add the frozen base embedding flow (ablation) | LaTexBlend · Customized generation | Concept alignment (SDINO): 0.4337 -> 0.6564 | Helped |
| Use varied prompt templates instead of fixed templates (ablation) | LaTexBlend · Customized generation | Textual alignment (ST_CLIP): 0.3539 -> 0.3684 | Helped |
| Use varied prompt templates instead of fixed templates (ablation) | LaTexBlend · Customized generation | Concept alignment (SI_CLIP): 0.7155 -> 0.8052 | Helped |
| Use varied prompt templates instead of fixed templates (ablation) | LaTexBlend · Customized generation | Concept alignment (SDINO): 0.5648 -> 0.6564 | Helped |
| Add blending guidance during inference (ablation) | LaTexBlend · Multi-concept customized generation | Concept fidelity | Helped |
| Use LaTexBlend instead of the evaluated baselines | Multi-concept customized generation · User study | Overall quality: 4.76 | Helped |
Surprising findings
- Blending guidance gives only subtle gains when there are few concepts (≤3); its benefit appears mainly as concept count grows.
- Without the frozen base flow, the method's concept fidelity degrades sharply, indicating the base flow is essential rather than optional.
Limitations
- Pre-trained T2I models struggle with lengthy, complex prompts describing many objects, limiting how many concepts can be scaled.
- Blending guidance requires gradient-based cross-attention optimization at inference, adding some cost for many concepts.
Experimental setup
Datasets: 30 subjects from prior studies (Gal et al., Kumari et al., Ruiz et al.)
Benchmarks: 30 customization subjects from prior studies
Metrics: CLIP text alignment (ST_CLIP) CLIP image similarity (SI_CLIP) DINO similarity (SDINO) user study ratings fine-tuning cost in hours
Baselines: Custom Diffusion CutMix Cones 2 Mix-of-Show OMG MuDI
Builds on: Custom Diffusion Latent Diffusion Models Mix-of-Show Cones 2 OMG
Closest papers by meaning
- MC^2: Multi-concept Guidance for Customized Multi-concept Generation (CVPR 2025 · similarity 0.90)
- Concept Weaver: Enabling Multi-Concept Fusion in Text-to-Image Models (CVPR 2024 · similarity 0.88)
- Non-confusing Generation of Customized Concepts in Diffusion Models (ICML 2024 · similarity 0.87)
- Concept Conductor: Orchestrating Multiple Personalized Concepts in Text-to-Image Synthesis (AAAI 2025 · similarity 0.87)
- MultiBooth: Towards Generating All Your Concepts in an Image from Text (AAAI 2025 · similarity 0.87)
- ConceptMix: A Compositional Image Generation Benchmark with Controllable Difficulty (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.