Research map / Image generation / Text-to-image diffusion
SerialGen: Personalized Image Generation by First Standardization Then Personalization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SerialGen, a two-stage tuning-free personalized human image generator that first standardizes a reference image's non-appearance elements (pose, background, expression, viewpoint) with an animation-based model trained on rendered 3D data, then personalizes an IP-Adapter on SDXL using standardized-reference/target pairs.
Problem
Tuning-free personalized text-to-image generation of human characters struggles to balance high text controllability with whole-body appearance consistency, because training pairs from the same source image cause the model to replicate the reference.
Approach
Train personalization on (standardized reference, target) pairs where both share appearance but differ in non-appearance elements, using a standardization model (animation-style ReferenceNet extended with foreground-background distinction and reference pose injection modules) trained on synthetic rendered 3D character data.
Key results
- Achieves CLIP-I 85.49 and CLIP-T 21.76 versus StoryMaker (84.24, 20.45) and IP-Adapter (81.31, 20.22); Face Sim. 0.53, second to StoryMaker's 0.54.
- Paired one-stage vs. two-stage: CLIP-I drops from 85.49 to 80.00 without standardization; AP-CLIP-I 77.74 vs. 83.17.
- Unpaired one-stage reaches CLIP-I 89.62 but CLIP-T only 16.08, indicating replication of the reference.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use unpaired one-stage training instead of SerialGen's two-stage standardized-reference training (ablation) | Personalized generation model · SerialGen test dataset | CLIP-T: 21.76 -> 16.08 | Hurt |
| Use unpaired one-stage training instead of SerialGen's two-stage standardized-reference training (ablation) | Personalized generation model · SerialGen test dataset | CLIP-I: 85.49 -> 89.62 | Mixed |
| Remove standardization and train paired one-stage (ablation) | Personalized generation model · SerialGen test dataset | CLIP-I: 85.49 -> 80.00 | Hurt |
| Remove standardization and train paired one-stage (ablation) | Personalized generation model · SerialGen outputs across prompts | AP-CLIP-I: 83.17 -> 77.74 | Hurt |
| Add FBDM to the standardization baseline (ablation) | Standardization model · Synthetic standardization test set | PSNR: 12.60 -> 12.79 | Helped |
| Add FBDM to the standardization baseline (ablation) | Standardization model · Synthetic standardization test set | SSIM: 0.5820 -> 0.5898 | Helped |
| Add RPIM to the baseline with FBDM (ablation) | Standardization model · Synthetic standardization test set | PSNR: 12.79 -> 12.84 | Helped |
| Add RPIM to the baseline with FBDM (ablation) | Standardization model · Synthetic standardization test set | SSIM: 0.5898 -> 0.5916 | Helped |
| Add FBDM and RPIM to the standardization model (ablation) | SerialGen · SerialGen test dataset | CLIP-I: 84.91 -> 85.16 | Helped |
| Add FBDM and RPIM to the standardization model (ablation) | SerialGen · SerialGen test dataset | CLIP-T: 21.71 -> 21.85 | Helped |
| Add FBDM and RPIM to the standardization model (ablation) | SerialGen · SerialGen test dataset | AP-CLIP-I: 82.86 -> 83.15 | Helped |
Surprising findings
- Unpaired one-stage training yields the highest CLIP-I (89.62) yet the lowest text alignment (CLIP-T 16.08), showing replication of the reference image.
Limitations
- Standardization model trained on synthetic 3D renders shows a style bias toward 3D style on real images.
- Face similarity is second to StoryMaker, which uses an extra face encoder.
- Evaluated on a small in-house test set of 40 characters.
Experimental setup
Datasets: synthetic rendered 3D character pairs (2,924 characters, 18 motion models) approximately 300,000 real character images TikTok
Benchmarks: in-house test set of 40 characters with 20 prompts TikTok
Metrics: CLIP-I CLIP-T Face Sim. (ArcFace) AP-CLIP-I PSNR SSIM FVD FID-VID user preference
Baselines: IP-Adapter FastComposer StoryMaker unpaired one-stage paired one-stage
Builds on: IP-Adapter Animate Anyone Stable Diffusion XL DWPose StoryDiffusion
Closest papers by meaning
- InstantBooth: Personalized Text-to-Image Generation without Test-Time Finetuning (CVPR 2024 · similarity 0.84)
- JeDi: Joint-Image Diffusion Models for Finetuning-Free Personalized Text-to-Image Generation (CVPR 2024 · similarity 0.84)
- Visual Persona: Foundation Model for Full-Body Human Customization (CVPR 2025 · similarity 0.84)
- FineStyle: Fine-grained Controllable Style Personalization for Text-to-image Models (NeurIPS 2024 · similarity 0.84)
- Rich Human Feedback for Text-to-Image Generation (CVPR 2024 · similarity 0.84)
- Tailored Visions: Enhancing Text-to-Image Generation with Personalized Prompt Rewriting (CVPR 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.