Research map / Image generation / Editing and style transfer
RealisID: Scale-Robust and Fine-Controllable Identity Customization via Local and Global Complementation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RealisID, a dual-branch ControlNet-based identity customization method on SDXL with a face-cropped, up-sampled local branch and a location-guided global branch, enabling small-face fidelity, fine pose/expression/location control, and multi-person generation trained only on single-person data.
Problem
Existing zero-shot identity customization methods fail to maintain identity fidelity for small faces, lack flexible fine control of face location, pose and expression, and do not scale to multi-person customization.
Approach
Two complementary ControlNet-style branches: a local branch that crops and bilinearly up-samples the face latent region (with pose-expression landmarks and relocation back to the face position) for scale-robust facial detail injection, and a global branch conditioned on a face-location mask for overall layout and harmony; multi-person generation is done by summing local injections and averaging gl
Key results
- On small faces, RealisID reaches FaceNet 0.767 and CLIP-I 0.701, versus 0.693 and 0.686 for the next best (InstantID), while winning three of four metrics.
- In the regular case RealisID is consistently second-best on all four metrics, behind PuLID or FlashFace on individual metrics.
- Multi-person FaceNet (0.788 at 1/4 size) is close to single-person (0.791), indicating identity is maintained when multiple persons are generated.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use RealisID instead of the strongest baseline on small faces | Identity customization · Small Face | FaceNet: 0.693 vs. 0.767 | Helped |
| Use RealisID instead of the strongest baseline on small faces | Identity customization · Small Face | CLIP-I: 0.701 vs. 0.686 | Helped |
| Use RealisID instead of the strongest baseline | Identity customization · Regular Case | ASP: 6.37 -> 6.22 | Hurt |
| Use RealisID instead of the strongest baseline | Identity customization · Regular Case | CLIP-T: 0.254 -> 0.234 | Hurt |
| Remove the local branch (ablation) | RealisID · Small Face | FaceNet: 0.767 -> 0.681 | Hurt |
| Add the local branch (ablation) | RealisID without local branch · Small Face | FaceNet: 0.681 -> 0.767 | Helped |
| Add the local branch (ablation) | RealisID without local branch · Small Face | CLIP-I: 0.673 -> 0.701 | Helped |
| Remove the global branch (ablation) | RealisID · Small Face | ASP: 6.11 -> 5.97 | Hurt |
| Add the global branch (ablation) | RealisID without global branch · Small Face | ASP: 5.97 -> 6.11 | Helped |
| Use RealisID instead of InstantID across the tested face sizes | Identity customization · Face Relative Size 1/4, 1/5, 1/6, and 1/7 | FaceNet: 0.765 vs. 0.791; 0.745 vs. 0.787; 0.708 vs. 0.772; 0.664 vs. 0.748 | Helped |
| Use RealisID rather than InstantID for pose control | Identity customization · Face Relative Size 1/4, 1/5, 1/6, and 1/7 | Pose control L1: 0.0313 -> 0.0187; 0.0316 -> 0.0194; 0.0333 -> 0.0205; 0.0339 -> 0.0231 | Helped |
| Use RealisID rather than InstantID for expression control | Identity customization · Face Relative Size 1/4, 1/5, 1/6, and 1/7 | Expression control L1: 0.2591 -> 0.1792; 0.2622 -> 0.1821; 0.2704 -> 0.1966; 0.2738 -> 0.2032 | Helped |
Limitations
- Multi-person customization cannot precisely control face-irrelevant areas such as clothing and hairstyle for each individual.
- Text encoders have weak spatial understanding, so spatial cues in prompts do not yield ideal per-person control.
Experimental setup
Datasets: CosmicMan CelebA-HQ
Benchmarks: CelebA-HQ (40 identities, 35 prompts)
Metrics: ASP CLIP-T FaceNet CLIP-I
Baselines: IP-Adapter-face-plus FlashFace PhotoMaker InstantID PuLID
Builds on: ControlNet Stable Diffusion IP-Adapter InstantID
Closest papers by meaning
- DreamSalon: A Staged Diffusion Framework for Preserving Identity-Context in Editable Face Generation (CVPR 2024 · similarity 0.86)
- Face2Diffusion for Fast and Editable Face Personalization (CVPR 2024 · similarity 0.86)
- PortraitBooth: A Versatile Portrait Model for Fast Identity-preserved Personalization (CVPR 2024 · similarity 0.85)
- PersonaMagic: Stage-Regulated High-Fidelity Face Customization with Tandem Equilibrium (AAAI 2025 · similarity 0.85)
- FaceA-Net: Facial Attribute-Driven ID Preserving Image Generation Network (AAAI 2025 · similarity 0.85)
- HiFi-Portrait: Zero-shot Identity-preserved Portrait Generation with High-fidelity Multi-face Fusion (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.