Research map / Image generation / Editing and style transfer
Text-conditional Attribute Alignment across Latent Spaces for 3D Controllable Face Image Synthesis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TcALign, which combines a text-conditional 3D editor (using an attribute embedding space) with a cross-modal latent mapper that predicts StyleGAN W-space corrections from 3D parameter differences, enabling text-driven control of expression, pose and illumination on face images.
Problem
Text-driven face manipulation (StyleCLIP-style) handles appearance attributes but lacks fine-grained control of 3D attributes such as pose and illumination, while 3D-aware face generators do not support text control.
Approach
Edit the 3D face representation (3DMM/SH/pose coefficients) under text guidance using an attribute embedding space that projects CLIP image embeddings onto text-related basis attributes, then map the resulting 3D difference into a StyleGAN W+ correction vector via a conditioned latent mapper with 3D and 2D landmark consistency losses.
Key results
- TcALign reports FID 7.104, IDD 0.107 and TARR expression 0.913 on 3,000 identities with 5 attributes each, beating StyleCLIP (7.595, 0.124, 0.728) and DeltaEdit (7.658, 0.119, 0.547).
- Against 3D-aware baselines, FID is far lower than DiscoFaceGAN (63.363) and DiffusionRig (27.634), and the paper reports better pose and illumination scores than DiffusionRig and GANControl.
- Ablations show removing the attribute embedding space, input w0, 3D priors, or correction-vector learning each degrades TARR, FID or IDD.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use TcALign instead of DiffusionRig | Face manipulation · Face manipulation evaluation (3,000-ID real face images; 5 attributes) | FID: 27.634 -> 7.104 | Helped |
| Use TcALign instead of StyleCLIP | Text-driven expression manipulation · Face manipulation evaluation (3,000-ID real face images; 5 attributes) | TARR: Expression: 0.728 -> 0.913 | Helped |
| Use TcALign instead of DeltaEdit | Text-driven expression manipulation · Face manipulation evaluation (3,000-ID real face images; 5 attributes) | TARR: Expression: 0.547 -> 0.913 | Helped |
| Add the attribute embedding space (AES) for disentangled text manipulation (ablation) | TcALign Text-conditional 3D Editor · 3D face manipulation | TARR: Pose | Helped |
| Condition the latent mapper on the source code w0 (ablation) | TcALign cross-modal latent mapper · 3D face manipulation | TARR: Pose | Helped |
| Add 3D priors to the framework (ablation) | TcALign · Face manipulation | FID | Helped |
| Predict a correction vector from Δθ instead of the global latent code (ablation) | TcALign cross-modal latent mapper · Face manipulation | IDD | Helped |
Surprising findings
- Global latent code prediction across modalities is reported as very difficult, with correction-vector learning being necessary for good results.
- StyleCLIP and DeltaEdit fail to control pose and illumination at all, despite strong expression control.
Limitations
- Pose and illumination control depends on the accuracy of the pretrained 3D predictor (Deep3DFR) and the rendering model.
- Relies on several frozen pretrained components (e4e, StyleGAN, CLIP, 3D predictor), so quality is bounded by them.
- Evaluated mainly on FFHQ-style face images with a limited set of attribute types.
Experimental setup
Datasets: FFHQ
Benchmarks: FFHQ
Metrics: FID IDD TARR (expression, illumination, pose)
Baselines: StyleCLIP DeltaEdit DiscoFaceGAN GANControl DiffusionRig
Builds on: StyleCLIP DeltaEdit e4e Deep3DFR
Closest papers by meaning
- Fast Text-to-3D-Aware Face Generation and Manipulation via Direct Cross-modal Mapping and Geometric Regularization (ICML 2024 · similarity 0.89)
- Text-Guided 3D Face Synthesis - From Generation to Editing (CVPR 2024 · similarity 0.87)
- Controllable 3D Face Generation with Conditional Style Code Diffusion (AAAI 2024 · similarity 0.86)
- CA-Edit: Causality-Aware Condition Adapter for High-Fidelity Local Facial Attribute Editing (AAAI 2025 · similarity 0.85)
- SDGAN: Disentangling Semantic Manipulation for Facial Attribute Editing (AAAI 2024 · similarity 0.85)
- 3D-Aware Face Editing via Warping-Guided Latent Direction Learning (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.