Research map / Image generation / Text-to-image diffusion
RealCustom: Narrowing Real Text Word for Real-Time Open-Domain Text-to-Image Customization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RealCustom, a real-time open-domain text-to-image customization method that narrows a real text word (e.g., "toy") to a given subject via adaptive mask guidance over cross-attention, avoiding pseudo-words and test-time optimization.
Problem
Pseudo-word customization methods suffer a dual-optimum paradox: subject similarity and text controllability cannot be simultaneously optimal because pseudo-words and the text share the same entire influence scope; existing methods also need test-time optimization or are limited to small object data
Approach
Instead of pseudo-words, progressively narrow a real text word's cross-attention region to the subject: a trained adaptive scoring module selects visual features for an added visual cross-attention, and at inference an adaptive mask (from aggregated T2I cross-attention) iteratively limits where subject information is injected in a text&image-to-image branch.
Key results
- RealCustom achieves CLIP-T 0.3204 vs 0.2967 for the best baseline BLIP-Diffusion, with zero test-time optimization steps.
- ImageReward improves to 0.8703 from 0.2690 (ELITE), a 223.5% relative gain claimed in the text.
- DINO-I reaches 0.7865 and CLIP-I 0.8552, the highest among compared methods (DreamBooth DINO-I 0.7851).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use RealCustom instead of the strongest baseline | Text-to-image customization · Customization evaluation | CLIP-T: 0.2967 -> 0.3204 | Helped |
| use RealCustom instead of the strongest baseline | Text-to-image customization · Customization evaluation | ImageReward: 0.2690 -> 0.8703 | Helped |
| use RealCustom instead of the strongest baseline | Text-to-image customization · Customization evaluation | CLIP-I: 0.8466 -> 0.8552 | Helped |
| use RealCustom instead of the strongest baseline | Text-to-image customization · Customization evaluation | DINO-I: 0.7851 -> 0.7865 | Helped |
| set γscope to 0.1 rather than 0.25 (ablation) | RealCustom · Customization evaluation | CLIP-I: 0.8552 -> 0.8085 | Hurt |
| set γscope to 0.2 rather than 0.25 (ablation) | RealCustom · Customization evaluation | CLIP-I: 0.8552 -> 0.8431 | Hurt |
| use binary masks instead of maximum normalization (ablation) | RealCustom · Customization evaluation | CLIP-T: 0.3204 -> 0.294 | Hurt |
| set γscope to 0.3 rather than 0.25 (ablation) | RealCustom · Customization evaluation | CLIP-T: 0.3204 -> 0.3129 | Hurt |
| set γscope to 0.4 rather than 0.25 (ablation) | RealCustom · Customization evaluation | CLIP-T: 0.3204 -> 0.3023 | Hurt |
| set γscope to 0.5 rather than 0.25 (ablation) | RealCustom · Customization evaluation | CLIP-T: 0.3204 -> 0.285 | Hurt |
| remove the adaptive scoring module (ablation) | RealCustom · Customization evaluation | CLIP-T: 0.3204 -> 0.3002 | Hurt |
| use textual score only (ablation) | RealCustom · Customization evaluation | CLIP-I: 0.8552 -> 0.8335 | Hurt |
Surprising findings
- Using all image features as visual conditions in training degrades both similarity and controllability at inference, attributed to a train-inference gap.
- Using image score alone for feature selection performs worse than textual score alone, since early generated features are noisy.
Limitations
- Relies on a real text word that describes the subject category (e.g., 'toy') to be narrowed down.
- Only a single reference image is used; no evaluation of multi-subject composition is reported.
Experimental setup
Datasets: LAION-5B (filtered subset)
Benchmarks: custom subject benchmark (not named)
Metrics: CLIP-T ImageReward CLIP-I DINO-I
Baselines: Textual Inversion DreamBooth Custom Diffusion ELITE BLIP-Diffusion
Builds on: Stable Diffusion DreamBooth Prompt-to-Prompt classifier-free guidance
Closest papers by meaning
- Direct Consistency Optimization for Robust Customization of Text-to-Image Diffusion models (NeurIPS 2024 · similarity 0.86)
- AttnDreamBooth: Towards Text-Aligned Personalized Text-to-Image Generation (NeurIPS 2024 · similarity 0.86)
- Large-Scale Text-to-Image Model with Inpainting is a Zero-Shot Subject-Driven Image Generator (CVPR 2025 · similarity 0.85)
- Navigating Text-To-Image Customization: From LyCORIS Fine-Tuning to Model Evaluation (ICLR 2024 · similarity 0.85)
- CtrlSynth: Controllable Image Text Synthesis for Data-Efficient Multimodal Learning (ICML 2025 · similarity 0.85)
- TextCtrl: Diffusion-based Scene Text Editing with Prior Guidance Control (NeurIPS 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.