Research map / Image generation / Editing and style transfer

Omni-ID: Holistic Identity Representation Designed for Generative Tasks

CVPR 2025 · Image generation › Editing and style transfer · approach: Diffusion face generation and editing · 3 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces Omni-ID, a fixed-size transformer face-identity representation trained with a few-to-many generative reconstruction objective and two decoders (masked transformer and flow-matching) on the new MFHQ video-face dataset.

Problem

Existing face identity representations (ArcFace, CLIP) are learned discriminatively from single images and lose nuanced, pose- and expression-spanning identity detail needed for generative face synthesis and personalization.

Approach

Learn a holistic identity encoder that consolidates a variable number of unstructured face images into a fixed set of learnable query tokens, trained generatively by reconstructing many unseen target views of the same person from a few inputs, with a multi-decoder objective combining masked reconstruction and conditional flow matching.

Key results

• On the MFHQ test set with 7 inputs, Omni-ID reaches ID similarity 0.742 versus 0.682 for CLIP and 0.535 for ArcFace in controllable face generation.
• Ablation removing MTD pretraining drops ID similarity from 0.708 to 0.468 (1 input), and removing few-to-many pretraining drops it to 0.616.
• Omni-ID's ID similarity improves monotonically with more input images (0.708 at 1 input to 0.742 at 7 inputs on MFHQ).

Measured effects reported in the paper

Surprising findings

• Flow-matching decoder alone is less effective for representation learning; the masked autoencoding objective is needed to learn a useful encoder.
• Using a 99% masking ratio slightly reduces performance, indicating an optimal mask ratio around 95%.

Limitations

• Omni-ID does not represent non-facial attributes such as hair, which can be hallucinated downstream.
• Training data has limited lighting diversity and skin-tone coverage, causing lighting injection and skin tone bias issues.
• Gains from additional input images are marginal beyond a few inputs; models trained with at most 3 inputs.

Experimental setup

Datasets: MFHQ WebFace21M WebFace42M CelebV-HQ VFHQ TalkingHead-1KH CelebV-Text internal licensed image dataset

Benchmarks: MFHQ test set WebFace test set (Webface21M-derived) personalized T2I test (10 identities, 20 prompts)

Metrics: ID similarity pose error

Baselines: ArcFace CLIP ArcFace + CLIP (FaceIDPlus-style) IP-Adapter FaRL ControlNet

Builds on: IP-Adapter CLIP ArcFace FLUX ControlNet

Closest papers by meaning

• FaceA-Net: Facial Attribute-Driven ID Preserving Image Generation Network (AAAI 2025 · similarity 0.88)
• Generative Unlearning for Any Identity (CVPR 2024 · similarity 0.86)
• FaithfulFaces: Pose-Faithful Facial Identity Preservation for Text-to-Video Generation (NeurIPS 2026 · similarity 0.85)
• Vec2Face: Scaling Face Dataset Generation with Loosely Constrained Vectors (ICLR 2025 · similarity 0.85)
• GIF: Generative Inspiration for Face Recognition at Scale (CVPR 2025 · similarity 0.85)
• UIFace: Unleashing Inherent Model Capabilities to Enhance Intra-Class Diversity in Synthetic Face Recognition (ICLR 2025 · similarity 0.85)