Research map / Image generation / Editing and style transfer
The Devil is in the Details: StyleFeatureEditor for Detail-Rich StyleGAN Inversion and High Quality Image Editing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes StyleFeatureEditor, a two-phase StyleGAN inversion encoder that predicts high-resolution F-space features for detail-rich reconstruction and adds a Feature Editor trained on e4e-generated edit pairs to keep edits working.
Problem
GAN inversion for StyleGAN faces a distortion-editability trade-off: W/W+ latents edit well but lose detail, while high-dimensional feature space F_k reconstructs well but loses editability.
Approach
Train an encoder to predict high-resolution (64x64) F_k features for reconstruction, then train a separate Feature Editor module that modifies F_k using the feature difference between edited and unedited W+ latents from a pretrained e4e encoder, with the edit supervision generated synthetically.
Key results
- On CelebA-HQ, SFE reaches LPIPS 0.019 and L2 0.002 versus 0.067 and 0.012 for Feature-Style and 0.076 and 0.013 for StyleRes, with FID 3.535 versus 8.505 for StyleRes.
- Editing FID is comparable to StyleRes (Smile 24.388 vs 24.465, Old 41.677 vs 43.698 for StyleRes) and the method runs in 0.070 s per image on a Tesla V100.
- On Stanford Cars, SFE attains LPIPS 0.039 and FID 4.035 versus 0.147 and 7.180 for Feature-Style.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use SFE instead of Feature-Style | StyleGAN inversion on CelebA-HQ · CelebA-HQ test | LPIPS: 0.067 -> 0.019 | Helped |
| use SFE instead of StyleRes | StyleGAN inversion on CelebA-HQ · CelebA-HQ test | LPIPS: 0.076 -> 0.019 | Helped |
| use SFE instead of StyleRes | StyleGAN editing on CelebA-HQ · Smile (-) | FID: 24.465 -> 24.388 | Helped |
| use SFE instead of StyleRes | StyleGAN editing on CelebA-HQ · Glasses (+) | FID: 73.089 -> 73.098 | No clear effect |
| use SFE instead of StyleRes | StyleGAN editing on CelebA-HQ · Old (+) | FID: 43.698 -> 41.677 | Helped |
| use SFE instead of Feature-Style | StyleGAN inversion on Stanford Cars · Stanford Cars test | LPIPS: 0.147 -> 0.039 | Helped |
| add Feature Editor H (ablation) | SFE on CelebA-HQ · CelebA-HQ test | LPIPS: 0.016 -> 0.019 | Hurt |
| add Feature Editor H (ablation) | SFE on CelebA-HQ · Smile (-) | FID: 28.149 -> 24.388 | Helped |
| add Feature Editor H (ablation) | SFE on CelebA-HQ · Old (+) | FID: 54.621 -> 41.677 | Helped |
| add Fuser (ablation) | SFE on CelebA-HQ · CelebA-HQ test | LPIPS: 0.023 -> 0.019 | Helped |
| include inversion loss during phase 2 (ablation) | SFE on CelebA-HQ · CelebA-HQ test | LPIPS: 0.037 -> 0.019 | Helped |
| use e4e instead of Inverter-predicted w (ablation) | SFE on CelebA-HQ · Old (+) | FID: 44.398 -> 41.677 | Helped |
Surprising findings
- Removing the Feature Editor improves inversion metrics slightly (LPIPS 0.016 vs 0.019) but breaks editing (Old FID rises to 54.6 from 41.7).
Limitations
- Editability is evaluated only with FID-based proxy metrics, since target edited images are unavailable.
- Some baseline checkpoints for the car domain are not public, so comparisons there are partial and Feature-Style was retrained by the authors.
Experimental setup
Datasets: FFHQ CelebA-HQ Stanford Cars
Benchmarks: CelebA-HQ Stanford Cars
Metrics: LPIPS L2 MS-SSIM FID inference time
Baselines: e4e pSp StyleTransformer ReStyle Padding Inverter HyperInverter HyperStyle HFGI Feature-Style StyleRes PTI
Builds on: Feature-Style StyleRes e4e StyleGAN2 PTI
Closest papers by meaning
- StyleRes: Transforming the Residuals for Real Image Editing With StyleGAN (CVPR 2023 · similarity 0.94)
- Delving StyleGAN Inversion for Image Editing: A Foundation Latent Space Viewpoint (CVPR 2023 · similarity 0.93)
- ReGANIE: Rectifying GAN Inversion Errors for Accurate Real Image Editing (AAAI 2023 · similarity 0.91)
- Gradual Residuals Alignment: A Dual-Stream Framework for GAN Inversion and Image Attribute Editing (AAAI 2024 · similarity 0.88)
- Tuning-Free Inversion-Enhanced Control for Consistent Image Editing (AAAI 2024 · similarity 0.87)
- HyperEditor: Achieving Both Authenticity and Cross-Domain Capability in Image Editing via Hypernetworks (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.