Research map / Image generation / Editing and style transfer
DreamSalon: A Staged Diffusion Framework for Preserving Identity-Context in Editable Face Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes DreamSalon, a noise-guided staged diffusion editing framework with frequency-based editing stage, gradient-based stochastic boosting stage, and covariance-guided prompt mixing for fine face attribute editing.
Problem
Identity fine editing: precisely modifying specific facial features of a subject while preserving its identity and context, which existing personalization and editing methods struggle with.
Approach
Uses frequency of predicted noises to define an aggressive editing stage and gradient of predicted noises to define a stochastic boosting stage, mixes source and target prompt embeddings with a time-varying weight, and modulates tokens with a covariance-difference map (CovDiff) to focus edits.
Key results
- On CelebA-HQ, DreamSalon reaches CLIP-I 0.837, CLIP-T 0.247 and DINO-I 0.958, the best of all compared methods.
- On FFE-Bench, DreamSalon gets CLIP-I 0.815 and DINO-I 0.932, ahead of PnP, EDICT and HyperDreambooth.
- DreamSalon fine-tunes 6x faster and needs 10x less storage than DreamBooth, with editing time of 26 s versus 14 s for DreamBooth.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add HyperDreambooth personalized weight generation (ablation) | DreamSalon · CelebA-HQ | CLIP-I: 0.778 vs 0.837 | Helped |
| add staged editing (ablation) | DreamSalon · CelebA-HQ | CLIP-I: 0.734 vs 0.837 | Helped |
| use adaptive frequency-guided λt instead of a prefixed value (ablation) | DreamSalon · CelebA-HQ | CLIP-I: 0.750 vs 0.837 | Helped |
| use stochastic rather than deterministic denoising in the boosting stage (ablation) | DreamSalon · CelebA-HQ | CLIP-I: 0.804 vs 0.837 | Helped |
| add covariance-guided prompt mixing (ablation) | DreamSalon · CelebA-HQ | CLIP-I: 0.802 vs 0.837 | Helped |
| optimize λt (ablation) | DreamSalon · CelebA-HQ | CLIP-I: 0.812 vs 0.837 | Helped |
| use DreamSalon rather than the strongest listed baseline | Face editing · CelebA-HQ | CLIP-I: 0.812 vs 0.837 | Helped |
| use DreamSalon rather than the strongest listed baseline | Face editing · FFE-Bench | CLIP-I: 0.808 vs 0.815 | Helped |
Surprising findings
- The text-to-image DreamBooth baseline has a very low DINO-I (0.150) on CelebA-HQ, while the editing-based baselines score much higher on that metric.
Limitations
- Editing time is longer than DreamBooth and HyperDreamBooth after fine-tuning.
- Stage boundaries and quantile thresholds (75%, 25%) are set by hand and tuned on examples; the ablation shows sensitivity to these choices.
Experimental setup
Datasets: CelebA-HQ FFHQ-unaligned
Benchmarks: CelebA-HQ FFE-Bench
Metrics: CLIP-I CLIP-T DINO-I fine-tuning time editing time
Baselines: DreamBooth HyperDreambooth LEDITS Prompt-to-Prompt Plug-and-Play EDICT ELITE Blended Latent Diffusion DiffDis
Builds on: DDIM HyperDreambooth Prompt-to-Prompt Plug-and-Play DreamBooth
Closest papers by meaning
- DreamIdentity: Enhanced Editability for Efficient Face-Identity Preserved Image Generation (AAAI 2024 · similarity 0.91)
- When StyleGAN Meets Stable Diffusion: a W+ Adapter for Personalized Image Generation (CVPR 2024 · similarity 0.89)
- FaceA-Net: Facial Attribute-Driven ID Preserving Image Generation Network (AAAI 2025 · similarity 0.88)
- PersonaMagic: Stage-Regulated High-Fidelity Face Customization with Tandem Equilibrium (AAAI 2025 · similarity 0.88)
- Energy-Guided Optimization for Personalized Image Editing with Pretrained Text-to-Image Diffusion Models (AAAI 2025 · similarity 0.87)
- PortraitBooth: A Versatile Portrait Model for Fast Identity-preserved Personalization (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.