Research map / Image generation / Editing and style transfer
ID-Patch: Robust ID Association for Group Photo Personalization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ID-Patch, which renders identity-specific patches at face locations as ControlNet input plus ID embeddings via cross-attention, enabling robust multi-ID group photo generation with position control.
Problem
Multi-identity group photo personalization suffers from ID leakage between people, leading to low face resemblance, wrong positions, and artifacts; existing methods rely on segmentation models, are slow, or leak identities.
Approach
Derive both a spatial ID patch (rendered on a black canvas at each nose-tip location and fed to ControlNet) and ID embeddings (appended to text embeddings) from the same face features, trained in two stages (patch-only first, then embeddings added), so each identity is bound to its location without segmentation masks or multiple denoising passes.
Key results
- ID-Patch achieves 0.751 ID resemblance and 0.958 association accuracy versus 0.644/0.926 for OMG+InstantID and 0.683/0.862 for InstantFamily.
- Generation time of 9.69 s per image versus 84.83 s for OMG+InstantID and 10.02 s for InstantFamily on an A100.
- Ablation shows removing patch projection drops association to 0.345 and single-stage training drops it to 0.784.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use ID-Patch instead of InstantFamily | Multi-ID group photo generation · Evaluation set | identity resemblance: 0.683 -> 0.751 | Helped |
| use ID-Patch instead of InstantFamily | Multi-ID group photo generation · Evaluation set | ID-position association accuracy: 0.862 -> 0.958 | Helped |
| use ID-Patch instead of OMG + InstantID | Multi-ID group photo generation · Evaluation set | generation time: 84.83 -> 9.69 | Helped |
| add ID patch projection (ablation) | ID-Patch · Evaluation set | ID-position association accuracy: 0.345 -> 0.958 | Helped |
| add ID token projection (ablation) | ID-Patch · Evaluation set | identity resemblance: 0.565 -> 0.751 | Helped |
| use two-stage training instead of single-stage training (ablation) | ID-Patch · Evaluation set | ID-position association accuracy: 0.784 -> 0.958 | Helped |
| set ID token injection ratio to 0.8 instead of 1.0 (ablation) | ID-Patch · Evaluation set | text alignment: 0.270 -> 0.273 | Helped |
Surprising findings
- Training end-to-end in a single stage fails to reliably localize IDs because the model prefers injecting identity into ID embeddings over patches, motivating the two-stage scheme.
Limitations
- Based on SDXL, which produces inaccuracies in body and finger anatomy.
- Facial features may entangle lighting and expression with identity, hurting generation under strong shadows.
- Performance gap remains between single-person and multi-person generation.
Experimental setup
Datasets: in-house single-person and multi-person image set (17M + 1.95M) Style Dataset (40 styles, pose and prompts) Identity Dataset (600 identities)
Benchmarks: Style Dataset Identity Dataset
Metrics: ID resemblance (FaceNet cosine similarity) ID-position association accuracy text alignment (CLIP cosine similarity) generation time (s)
Baselines: OMG + InstantID InstantFamily
Builds on: ControlNet InstantFamily SDXL Perceiver Resampler InstantID
Closest papers by meaning
- FaceA-Net: Facial Attribute-Driven ID Preserving Image Generation Network (AAAI 2025 · similarity 0.86)
- PhotoMaker: Customizing Realistic Human Photos via Stacked ID Embedding (CVPR 2024 · similarity 0.85)
- When StyleGAN Meets Stable Diffusion: a W+ Adapter for Personalized Image Generation (CVPR 2024 · similarity 0.85)
- HiFi-Portrait: Zero-shot Identity-preserved Portrait Generation with High-fidelity Multi-face Fusion (CVPR 2025 · similarity 0.85)
- FaithfulFaces: Pose-Faithful Facial Identity Preservation for Text-to-Video Generation (NeurIPS 2026 · similarity 0.84)
- RealisID: Scale-Robust and Fine-Controllable Identity Customization via Local and Global Complementation (AAAI 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.