Research map / Image generation / Editing and style transfer
CA-Edit: Causality-Aware Condition Adapter for High-Fidelity Local Facial Attribute Editing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CA-Edit, an inpainting-based local facial attribute editor with a new attribute-caption dataset (LAMask-Caption), a Causality-Aware Condition Adapter (CA²) that balances image and text conditions, and Skin Transition Frequency Guidance (STFG) for boundary smoothness.
Problem
Text-guided local facial attribute editing via inpainting suffers from poor alignment with local attribute descriptions and loss of facial skin details, plus artifacts at mask boundaries, and existing datasets lack local facial attribute captions.
Approach
Combines a dataset of local facial attribute captions generated by MLLMs with face parsing masks, a reference-net adapter that injects original-image skin detail via an additional attention branch gated by a learned text-dependent score map, and low-frequency guidance on boundary regions identified from text cross-attention during sampling.
Key results
- Ours achieves LPIPS 0.085 versus 0.160 for the best prior baseline (InstructPix2Pix) and FID/L-FID 4.81/1.99, the second-best FID behind SD Inpainting (3.11/1.61).
- User study: Ours is preferred over SD Inpainting, BrushNet, InstructPix2Pix, DiffusionCLIP, Asyrp and StyleCLIP on face fidelity, text-attribute consistency and human preference, with most comparisons above 50%.
- Ours obtains the highest CLIP text score across FaRL, FLIP and CLIP (e.g., 23.58 on FaRL), while SD Inpainting has the highest CLIP image score.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CA-Edit instead of SD Inpainting | Local facial attribute editing · FFLEBench | FID: 3.11 -> 4.81 | Hurt |
| Use CA-Edit instead of InstructPix2Pix | Local facial attribute editing · FFLEBench | LPIPS: 0.160 -> 0.085 | Helped |
| Use CA-Edit instead of InstructPix2Pix | Local facial attribute editing · FFLEBench | HPSv2: 0.263 -> 0.264 | Helped |
| Use CA-Edit instead of DiffusionClip | Local facial attribute editing · FFLEBench | ID: 0.73 -> 0.72 | Hurt |
| Use CA-Edit instead of the strongest listed baseline | Local facial attribute editing · FFLEBench | CS Text (FaRL): 22.32 -> 23.58 | Helped |
| Use CA-Edit instead of InstructPix2Pix | Local facial attribute editing · FFLEBench | CS Text (FLIP): 20.39 -> 20.26 | Hurt |
| Use CA-Edit instead of the strongest listed baseline | Local facial attribute editing · FFLEBench | CS Text (CLIP): 16.45 -> 16.88 | Helped |
| Use CA-Edit instead of SD Inpainting | Local facial attribute editing · FFLEBench | CS Image (CLIP): 95.24 -> 92.67 | Hurt |
| Add CA² to the method (ablation) | CA-Edit · FFLEBench | FID / Local-FID: 4.13 -> 4.81 | Helped |
| Add CA² to the method (ablation) | CA-Edit · FFLEBench | LPIPS: 0.138 -> 0.085 | Helped |
| Add CA² to the method (ablation) | CA-Edit · FFLEBench | MPS: 1.06 -> / | Hurt |
| Add spatial control of visual cross-attention (ablation) | CA-Edit · FFLEBench | LPIPS: 0.153 -> 0.085 | Helped |
Surprising findings
- SD Inpainting achieves the lowest FID despite neglecting text prompts, so FID alone rewards ignoring the edit.
- ID similarity can conflict with editing goals, since better attribute alignment may damage identity cues such as eyebrows.
Limitations
- Relies on coarse masks; fine-grained masks produce boundary artifacts, so boundary handling depends on STFG.
- ID similarity is not a reliable fidelity measure for local attribute editing.
- Dataset captions are generated by MLLMs, whose quality is not validated beyond the described pipeline.
Experimental setup
Datasets: FaceCaption-15M FFHQ CelebMask-HQ CelebA-Dialog
Benchmarks: FFLEBench
Metrics: FID Local-FID LPIPS ID similarity MPS HPSv2 CLIP score (CS Text / CS Image) user study (FF, TAC, HP)
Baselines: SD Inpainting BrushNet InstructPix2Pix DiffusionClip Asyrp StyleClip Blended Diffusion Renoise Inversion
Builds on: ControlNet BrushNet IP-Adapter Blended Diffusion Stable Diffusion
Closest papers by meaning
- SDGAN: Disentangling Semantic Manipulation for Facial Attribute Editing (AAAI 2024 · similarity 0.86)
- RetouchGPT: LLM-based Interactive High-Fidelity Face Retouching via Imperfection Prompting (AAAI 2025 · similarity 0.85)
- Text-Guided 3D Face Synthesis - From Generation to Editing (CVPR 2024 · similarity 0.85)
- DreamSalon: A Staged Diffusion Framework for Preserving Identity-Context in Editable Face Generation (CVPR 2024 · similarity 0.85)
- Text-conditional Attribute Alignment across Latent Spaces for 3D Controllable Face Image Synthesis (CVPR 2024 · similarity 0.85)
- Improving Editability in Image Generation with Layer-wise Memory (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.