Research map / Image generation / Text-to-image diffusion
MagicNaming: Consistent Identity Generation by Finding a “Name Space” in T2I Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Learns an image-to-"name embedding" encoder (CLIP-L/G features plus MLP) trained on a LAION-derived celebrity name-embedding dataset (LaionCele), so SDXL can generate consistent identities from a predicted name token without fine-tuning the U-Net.
Problem
Text-to-image diffusion models generate consistent identities for celebrities from their names, but generic identities lack such a name; fine-tuning-based identity personalization methods degrade the generator's semantic and scene-generation abilities.
Approach
Find a "Name Space" of text embeddings: an image encoder trained on (face image, name-embedding) pairs from LAION celebrity captions predicts a name embedding that is prepended to the prompt embedding, with mean-shift guidance and norm rescaling, leaving the diffusion model's weights untouched.
Key results
- Overall CLIP-TI of 38.24 for the proposed method vs 36.29 for SDXL, 30.36 for IP-Adapter and 36.78 for PhotoMaker.
- Overall FID vs SDXL of 56.75, versus 110.73 for IP-Adapter and 95.16 for PhotoMaker.
- ID consistency of 0.5624 vs 0.5466 for PhotoMaker and 0.3365 for SDXL on the emotion-editing task.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use MagicNaming name embeddings without fine-tuning | SDXL v1.0 · Scene construction | CLIP-TI: 43.45 vs 40.72 | Helped |
| Use MagicNaming name embeddings without fine-tuning | SDXL v1.0 · All tasks | FID with SDXL: 95.16 -> 56.75 | Helped |
| Use MagicNaming name embeddings without fine-tuning | SDXL v1.0 · Emotional editing | ID consistency: 0.5466 -> 0.5624 | Helped |
| Fine-tune with LoRA on LaionCele instead of leaving the generator unfine-tuned (ablation) | MagicNaming on SDXL v1.0 · Scene construction | CLIP-TI: 43.45 -> 37.54 | Hurt |
| Fine-tune with LoRA on LaionCele instead of leaving the generator unfine-tuned (ablation) | MagicNaming on SDXL v1.0 · Scene construction | Reference similarity: 0.4008 -> 0.5004 | Helped |
| Fine-tune with LoRA on LaionCele instead of leaving the generator unfine-tuned (ablation) | MagicNaming on SDXL v1.0 · Scene construction | ID consistency: 0.4407 -> 0.6236 | Helped |
| Fine-tune the U-Net on LaionCele instead of leaving the generator unfine-tuned (ablation) | MagicNaming on SDXL v1.0 · Scene construction | CLIP-TI: 43.45 -> 32.55 | Hurt |
| Fine-tune the U-Net on LaionCele instead of leaving the generator unfine-tuned (ablation) | MagicNaming on SDXL v1.0 · Scene construction | ID consistency: 0.4407 -> 0.6631 | Helped |
| Fine-tune with LoRA on Ins700 instead of leaving the generator unfine-tuned (ablation) | MagicNaming on SDXL v1.0 · Scene construction | CLIP-TI: 43.45 -> 40.66 | Hurt |
| Fine-tune with LoRA on Ins700 instead of leaving the generator unfine-tuned (ablation) | MagicNaming on SDXL v1.0 · Scene construction | Reference similarity: 0.4008 -> 0.4816 | Helped |
| Fine-tune the U-Net on Ins700 instead of leaving the generator unfine-tuned (ablation) | MagicNaming on SDXL v1.0 · Scene construction | CLIP-TI: 43.45 -> 36.51 | Hurt |
| Fine-tune the U-Net on Ins700 instead of leaving the generator unfine-tuned (ablation) | MagicNaming on SDXL v1.0 · Scene construction | ID consistency: 0.4407 -> 0.6356 | Helped |
Surprising findings
- Fine-tuning the U-Net yielded higher ID consistency but lowered CLIP-TI, showing a trade-off between identity and generative capability.
- Embeddings from the generated (SDXL) images gave a lower-quality encoder than LaionCele-trained ones, suggesting a domain gap between generated and real images.
Limitations
- Test set is small (20 reference images and 40 prompts), and reference images were selected to be ones SDXL could not identify by name.
- Name embeddings keep only the first 8 dimensions of the text embedding, which may limit identity detail.
- The LaionCele dataset is built with heavy use of SDXL and ChatGPT in its pipeline, so the labels depend on automatic filtering.
Experimental setup
Datasets: LAION-5B LaionCele Ins700
Benchmarks: 40-prompt four-task test set (scene, style, action, emotion)
Metrics: CLIP-TI score FID vs SDXL outputs ID consistency (FaceNet) reference similarity
Baselines: SDXL IP-Adapter PhotoMaker
Builds on: Stable Diffusion XL classifier-free guidance DDIM
Closest papers by meaning
- When StyleGAN Meets Stable Diffusion: a W+ Adapter for Personalized Image Generation (CVPR 2024 · similarity 0.86)
- DreamIdentity: Enhanced Editability for Efficient Face-Identity Preserved Image Generation (AAAI 2024 · similarity 0.85)
- FaceA-Net: Facial Attribute-Driven ID Preserving Image Generation Network (AAAI 2025 · similarity 0.83)
- Interpreting the Weight Space of Customized Diffusion Models (NeurIPS 2024 · similarity 0.83)
- Origin Identification for Text-Guided Image-to-Image Diffusion Models (ICML 2025 · similarity 0.83)
- JeDi: Joint-Image Diffusion Models for Finetuning-Free Personalized Text-to-Image Generation (CVPR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.