Research map / Image generation / Editing and style transfer
Spatial-Contextual Discrepancy Information Compensation for GAN Inversion
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SDIC, a GAN inversion method that predicts a spatial-contextual discrepancy map between the original and e4e-reconstructed images and uses it to compensate both the StyleGAN latent code and an early generator layer before editing ("compensate-and-edit").
Problem
GAN inversion suffers from a distortion-editability trade-off: W-space inversion is editable but loses image detail, while higher-fidelity methods degrade editability or introduce artifacts.
Approach
A discrepancy information prediction network (two-branch spatial-contextual hourglass plus discrepancy hourglass with spatial attention fusion) predicts a discrepancy map from the original and initial reconstruction; a compensation network applies affine compensation to the latent code and nonlinear attention-based compensation to generator layer 7, and editing adds the edit difference to the enha
Key results
- On CelebA-HQ inversion (first 1,000 images), SDIC reaches PSNR 27.672 and LPIPS 0.057, versus 25.665 and 0.083 for the next-best CLCAE.
- SDIC achieves ID 0.871 on inversion and a 65.832% user-study preference in face editing, versus 22.918% for the next-best method.
- SDIC inference takes 0.321 s per image, far faster than PTI (201.357 s), though slower than e4e (0.033 s).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SDIC instead of PTI for inversion | StyleGAN inversion on CelebA-HQ · CelebA-HQ inversion | SSIM: 0.703 -> 0.815 | Helped |
| Use SDIC instead of PTI for editing | StyleGAN editing on CelebA-HQ · CelebA-HQ editing | User Study: 22.500% -> 65.832% | Helped |
| Add the two-branch spatial-contextual hourglass module (ablation) | SDIC inversion · Face domain | reconstruction fidelity and artifacts | Helped |
| Add spatial attention fusion instead of conventional affine transformation for latent-map compensation (ablation) | SDIC · Face domain | image quality | Helped |
| Use compensate-and-edit rather than edit-and-compensate (ablation) | SDIC editing · Face domain | editability | Helped |
| Add spatial attention fusion (ablation) | SDIC inversion · Face domain | L2: 0.009 -> 0.007 | Helped |
| Add spatial attention fusion (ablation) | SDIC inversion · Face domain | LPIPS: 0.062 -> 0.057 | Helped |
| Add spatial attention fusion (ablation) | SDIC inversion · Face domain | ID (ArcFace): 0.929 -> 0.948 | Helped |
| Add spatial-contextual information and attention fusion (ablation) | SDIC inversion · Face domain | L2: 0.020 -> 0.007 | Helped |
| Compensate generator layer 9 rather than layer 7 (ablation) | SDIC inversion · Face domain | L2: 0.007 -> 0.004 | Helped |
| Compensate generator layer 9 rather than layer 7 (ablation) | SDIC editing · Face domain | editing artifacts | Hurt |
| Compensate generator layer 7 rather than layer 5 (ablation) | SDIC inversion · Face domain | L2: 0.176 -> 0.007 | Helped |
Surprising findings
- Compensating later generator layers (9) improves fidelity further but increases overfitting and visible editing artifacts, so layer 7 was chosen as a trade-off.
Limitations
- Struggles with large manipulations such as significant viewpoint changes, producing artifacts due to misalignment between reconstructed and original images.
Experimental setup
Datasets: FFHQ CelebA-HQ Stanford Cars
Benchmarks: CelebA-HQ Stanford Cars
Metrics: ID SSIM PSNR LPIPS L2 inference time user study
Baselines: PTI e4e HyperStyle HFGI FS CLCAE
Builds on: e4e HFGI StyleGAN InterFaceGAN GANSpace
Closest papers by meaning
- High-Fidelity GAN Inversion for Image Attribute Editing (CVPR 2022 · similarity 0.90)
- ReGANIE: Rectifying GAN Inversion Errors for Accurate Real Image Editing (AAAI 2023 · similarity 0.89)
- Spatially-Adaptive Multilayer Selection for GAN Inversion and Editing (CVPR 2022 · similarity 0.88)
- Gradual Residuals Alignment: A Dual-Stream Framework for GAN Inversion and Image Attribute Editing (AAAI 2024 · similarity 0.87)
- WarpGAN: Warping-Guided 3D GAN Inversion with Style-Based Novel View Inpainting (NeurIPS 2025 · similarity 0.86)
- In-N-Out: Faithful 3D GAN Inversion with Volumetric Decomposition for Face Editing (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.