Research map / Image generation / Text-to-image diffusion
FreeCustom: Tuning-Free Customized Image Generation for Multi-Concept Composition
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FreeCustom, a tuning-free multi-concept customized image generator that uses multi-reference self-attention with weighted concept masks on Stable Diffusion v1.5, using one image per concept.
Problem
Customized text-to-image generation of multiple user-specified concepts usually requires per-concept fine-tuning or large-scale retraining, which is slow and struggles with identity preservation when combining concepts from one-shot references.
Approach
Run a concept-reference path that extracts query/key/value features from reference images with the same diffusion U-Net, then inject them into self-attention of selected deep blocks via multi-reference self-attention, with concept masks and per-concept weights that restrict attention to the referenced regions. Context-rich reference images (concepts shown in use) are found to preserve identity bet
Key results
- Multi-concept: CLIP-T 33.78 vs 29.07 (Custom Diffusion) and 22.14 (Perfusion); DINOv2 0.7625 vs 0.6545 for Custom Diffusion.
- Single-concept: CLIP-T 32.02 vs 29.13 for BLIP Diffusion, but DINOv2 0.8376 is below DreamBooth's 0.8948.
- Inference takes 36s for 2 concepts and 58s for 3 concepts with zero preprocessing, versus 300s to 835s preprocessing for multi-concept tuning baselines.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FreeCustom instead of BLIP Diffusion | single-concept customization · single-concept customization | CLIP-T: 29.1278 -> 32.0206 | Helped |
| Use FreeCustom instead of BLIP Diffusion | single-concept customization · single-concept customization | CLIP-IQA: 0.7655 -> 0.7292 | Hurt |
| Use FreeCustom instead of Custom Diffusion | multi-concept composition · multi-concept composition | CLIP-T: 29.0702 -> 33.7826 | Helped |
| Use FreeCustom instead of Custom Diffusion | multi-concept composition · multi-concept composition | CLIP-IQA: 0.8921 -> 0.9002 | Helped |
| Add weighted masks to MRSA (ablation) | FreeCustom · multi-concept composition | concept identity preservation | Helped |
| Replace self-attention with MRSA only in blocks [5, 6], rather than all 7 blocks (ablation) | FreeCustom · multi-concept composition | generation quality | Helped |
| Provide reference images with context interactions rather than isolated concept images (ablation) | FreeCustom · multi-concept composition | concept fidelity | Helped |
Surprising findings
- Image similarity (DINOv2) for single-concept customization is lower than DreamBooth, the authors attribute this to a trade-off with text alignment.
Limitations
- Lacks an explicit module to perceive the structure of input reference concepts.
- Reference images need concepts shown in context; isolated objects degrade results.
- Single-concept image similarity (DINOv2/CLIP-I) is mediocre compared to fine-tuning methods.
Experimental setup
Datasets: Internet-sourced reference images curated from customization and editing studies
Benchmarks: custom concept images (single and multi-concept, curated)
Metrics: DINOv2 CLIP-I CLIP-T CLIP-T-L CLIP-IQA inference time user study
Baselines: DreamBooth Textual Inversion NeTI BLIP Diffusion Custom Diffusion Perfusion
Builds on: Stable Diffusion MasaCtrl Grounded-Segment-Anything
Closest papers by meaning
- MC^2: Multi-concept Guidance for Customized Multi-concept Generation (CVPR 2025 · similarity 0.91)
- ConceptMix: A Compositional Image Generation Benchmark with Controllable Difficulty (NeurIPS 2024 · similarity 0.88)
- Concept Weaver: Enabling Multi-Concept Fusion in Text-to-Image Models (CVPR 2024 · similarity 0.88)
- Non-confusing Generation of Customized Concepts in Diffusion Models (ICML 2024 · similarity 0.88)
- Finetuning LLMs for Automatic Concept to TTI Prompt Generation (Student Abstract) (AAAI 2024 · similarity 0.87)
- Concept Conductor: Orchestrating Multiple Personalized Concepts in Text-to-Image Synthesis (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.