Research map / Image generation / Editing and style transfer
FaceA-Net: Facial Attribute-Driven ID Preserving Image Generation Network
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FaceA-Net, a test-time fine-tuning method for ID-preserving image generation that uses attribute-level reconstruction losses and decouples context generation (SDXL) from face generation (an SD inpainting model with LoRA).
Problem
ID-preserving image generation struggles to achieve both high identity fidelity (especially fine-grained facial attribute details) and high-quality context/prompt alignment, since test-time tuning or added adapters degrade the base model's generative ability.
Approach
An attribute-driven training method adds a global face reconstruction task and local tasks for eyes, nose, and mouth, with learnable [feat] token embeddings; an ID-context decoupling framework generates the scene with a large model (SDXL), then detects the face and inpaints it with a lightly fine-tuned inpainting model so fine-tuning never touches the context generator.
Key results
- FaceA-Net achieves the highest Face Sim. (70.7%) among compared methods, versus InstantID 70.0%, IP-Adapter 62.7%, Photo Maker 56.3%, and Celeb Basis 45.9%.
- CLIP-T of 23.3% is second only to InstantID (23.8%), which was given the pose condition from FaceA-Net's context images; without that condition InstantID's CLIP-T drops to 19.2.
- Ablation: removing the attribute-driven loss and embeddings lowers Face Sim. from 70.7 to 67.8 (loss only gives 69.8 with embeddings removed).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add attribute-driven reconstruction tasks for the whole face and local facial attributes (ablation) | FaceA-Net · Evaluation dataset | Face Sim.: 67.8 -> 70.7 | Helped |
| Add attribute-driven feature embeddings (ablation) | FaceA-Net · Evaluation dataset | Face Sim.: 69.8 -> 70.7 | Helped |
| Add attribute-driven training (reconstruction loss and feature embeddings) (ablation) | FaceA-Net · Evaluation dataset | CLIP-T: 23.4 -> 23.3 | No clear effect |
| Add attribute-driven feature embeddings (ablation) | FaceA-Net · Evaluation dataset | CLIP-T: 23.4 -> 23.3 | No clear effect |
Surprising findings
- Without a pose condition, InstantID's CLIP-T falls to 19.2, showing zero-shot ID methods can ignore the prompt.
Limitations
- Requires per-identity test-time fine-tuning rather than zero-shot generation.
- Evaluation uses a small self-constructed set of 28 IDs and 40 prompts.
- Relies on an external detector (Grounding-DINO) and a separate LLM-based prompt split that is not part of the contribution.
Experimental setup
Datasets: Evaluation set of 28 IDs with 3 images each and 40 prompts (constructed by authors)
Benchmarks: Authors' 28-ID evaluation set with 40 prompts
Metrics: CLIP-T CLIP-I DINO Face Sim. user study votes on ID fidelity, aesthetics, prompt alignmen
Baselines: Celeb Basis IP-Adapter Photo Maker InstantID
Builds on: Stable Diffusion SDXL DreamBooth Textual Inversion Grounding-DINO
Closest papers by meaning
- Omni-ID: Holistic Identity Representation Designed for Generative Tasks (CVPR 2025 · similarity 0.88)
- When StyleGAN Meets Stable Diffusion: a W+ Adapter for Personalized Image Generation (CVPR 2024 · similarity 0.88)
- DreamSalon: A Staged Diffusion Framework for Preserving Identity-Context in Editable Face Generation (CVPR 2024 · similarity 0.88)
- Generative Unlearning for Any Identity (CVPR 2024 · similarity 0.87)
- $\text{ID}^3$: Identity-Preserving-yet-Diversified Diffusion Models for Synthetic Face Recognition (NeurIPS 2024 · similarity 0.87)
- Vec2Face: Scaling Face Dataset Generation with Loosely Constrained Vectors (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.