Research map / Image generation / Editing and style transfer
SyFormer: Structure-Guided Synergism Transformer for Large-Portion Image Inpainting
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SyFormer, a structure-guided dual-branch transformer for large-portion image inpainting that uses structure maps from a structure-aware generator, with dual-routing filtering, structurally compact perception, and feature decoding alignment modules.
Problem
Image inpainting of large missing regions in high-resolution images, where long-range dependencies are distorted by cluttered backgrounds and reference information is scarce.
Approach
Maps structure priors (from a structure-aware generator) into an affinity matrix via a lightweight self-attention (SCP) guiding patch matching, runs a parallel transformer branch on the corrupted image with region-level routing filtering (DRF) to suppress mask noise, and fuses multi-level features with a feature decoding alignment block.
Key results
- On Places2 at 30-40% mask ratio, SyFormer reaches PSNR 24.0885 and SSIM 0.8552, outperforming the listed baselines in the table.
- On CelebA-HQ at 50-60% mask ratio, SSIM 0.8001 and LPIPS 0.1640, the best among compared methods in the table.
- Averaged over both datasets, PSNR improves by 0.7587 dB over AOT, 1.0472 dB over Lama, 1.7247 dB over MAT, 1.3354 dB over MADF, 0.9868 dB over Big-Lama and 0.3258 dB over Coordfill.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SyFormer instead of the strongest listed baseline, Coordfill | Image inpainting · CelebA-HQ, mask 30-40% | PSNR: 27.5136 -> 27.9467 | Helped |
| Use SyFormer instead of the strongest listed baseline, Coordfill | Image inpainting · CelebA-HQ, mask 40-50% | PSNR: 25.5431 -> 26.0252 | Helped |
| Use SyFormer instead of the strongest listed baseline, Coordfill | Image inpainting · CelebA-HQ, mask 50-60% | PSNR: 22.4907 -> 22.6334 | Helped |
| Use SyFormer instead of the strongest listed baseline, Coordfill | Image inpainting · Places2, mask 30-40% | PSNR: 23.7492 -> 24.0885 | Helped |
| Use SyFormer instead of the strongest listed baseline, Coordfill | Image inpainting · Places2, mask 40-50% | PSNR: 22.0542 -> 22.3871 | Helped |
| Use SyFormer instead of the strongest listed baseline, Coordfill | Image inpainting · Places2, mask 50-60% | PSNR: 19.7105 -> 19.9352 | Helped |
| Add DRF (ablation) | SyFormer · Places2, all reported mask ratios | PSNR: 23.3944 -> 24.0885; 21.7255 -> 22.3871; 19.3818 -> 19.9352 | Helped |
| Add FDA (ablation) | SyFormer · Places2, all reported mask ratios | PSNR: 24.0317 -> 24.0885; 22.3508 -> 22.3871; 19.8796 -> 19.9352 | Helped |
| Add SCP (ablation) | SyFormer · Places2, all reported mask ratios | PSNR: 23.4699 -> 24.0885; 21.8667 -> 22.3871; 19.6183 -> 19.9352 | Helped |
| Add DRF (ablation) | SyFormer · CelebA-HQ, all reported mask ratios | PSNR: 27.1105 -> 27.9467; 25.1704 -> 26.0252; 22.2953 -> 23.1334 | Helped |
| Add FDA (ablation) | SyFormer · CelebA-HQ, all reported mask ratios | PSNR: 27.8494 -> 27.9467; 25.9098 -> 26.0252; 22.9472 -> 23.1334 | Helped |
| Add SCP (ablation) | SyFormer · CelebA-HQ, all reported mask ratios | PSNR: 26.5621 -> 27.9467; 24.9421 -> 26.0252; 21.7414 -> 23.1334 | Helped |
Limitations
- Evaluated only on two datasets (CelebA-HQ and Places2) with irregular masks.
- Relies on a structure-aware generator whose quality bounds the structural prior.
Experimental setup
Datasets: CelebA-HQ Places2
Benchmarks: CelebA-HQ Places2
Metrics: SSIM PSNR MAE LPIPS
Baselines: AOT Lama MAT MADF Coordfill Big-Lama
Builds on: Contextual Attention Lama MAT EdgeConnect Deformable Downsample Update Module
Closest papers by meaning
- Reconstructing the Image Stitching Pipeline: Integrating Fusion and Rectangling into a Unified Inpainting Model (NeurIPS 2024 · similarity 0.86)
- Towards Enhanced Image Inpainting: Mitigating Unwanted Object Insertion and Preserving Color Consistency (CVPR 2025 · similarity 0.85)
- VSFormer: Visual-Spatial Fusion Transformer for Correspondence Pruning (AAAI 2024 · similarity 0.84)
- Paint by Inpaint: Learning to Add Image Objects by Removing Them First (CVPR 2025 · similarity 0.84)
- DesignEdit: Unify Spatial-Aware Image Editing via Training-free Inpainting with a Multi-Layered Latent Diffusion Framework (AAAI 2025 · similarity 0.84)
- WaveFormer: Wavelet Transformer for Noise-Robust Video Inpainting (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.