Research map / Image generation / Image restoration
Image Harmonization With Transformer
Zonghui Guo, Dongsheng Guo, Haiyong Zheng, Zhaorui Gu, Bing Zheng, Junyu Dong
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace CNN harmonizer with Transformer-encoder HT | E-D CNN · iHarmony4 (all) | fMSE: 625.67 -> 395.66 | Improved |
| add disentanglement (reflectance/illumination) with Transformers | HT · iHarmony4 (all) | fMSE: 395.66 -> 320.78 | Improved |
| increase token number from N to 16N (stride 8 to 2), FC embedding (ablation) | Transformer on iHarmony4 · iHarmony4 (all), 1-head 3-layer TRE | fMSE: 611.25 -> 447.64 | Improved |
| change embedding type from FC to CNN at T=4N (ablation) | Transformer on iHarmony4 · iHarmony4 (all), 1-head 3-layer TRE | fMSE: 522.84 -> 520.19 | No clear effect |
| increase TRE layers from 3 to 9 (1 head) (ablation) | HT · iHarmony4 (all) | fMSE: 502.37 -> 415.60 | Improved |
| increase TRE layers from 9 to 12 (1 head) (ablation) | HT · iHarmony4 (all) | fMSE: 415.60 -> 419.08 | No clear effect |
| increase attention heads from 1 to 4 at 9 layers (ablation) | HT · iHarmony4 (all) | fMSE: 415.60 -> 392.74 | Improved |
| add Transformer decoder layers to encoder (E(3)+D(6) vs E(9)) (ablation) | HT · iHarmony4 (all) | fMSE: 415.60 -> 403.76 | No clear effect |
| use Transformer for reflectance pathway instead of CNN (ablation) | D-HT · iHarmony4 (all) | fMSE: 479.94 -> 352.55 | Improved |
| use Transformer for illumination pathway instead of CNN (ablation) | D-HT · iHarmony4 (all) | fMSE: 479.94 -> 348.80 | Improved |
| use Transformers in both pathways (ablation) | D-HT · iHarmony4 (all) | fMSE: 479.94 -> 320.78 | Improved |
| apply HT to inpainting vs RFR-Net | HT · Paris StreetView inpainting | PSNR: 28.42 -> 29.55 | Improved |
Earlier methods this paper compares with
Named baselines from earlier accepted papers in this paper's results tables: better than 2 of 2 on most shared results.
- DoveNet — DoveNet: Deep Image Harmonization via Domain Verification (CVPR 2020) · better on 4 of 4 shared results
- DeepLPF — DeepLPF: Deep Local Parametric Filters for Image Enhancement (CVPR 2020) · better on 1 of 1 shared result
Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.
Closest papers by meaning
- Progressive Painterly Image Harmonization from Low-Level Styles to High-Level Styles (AAAI 2024 · similarity 0.88)
- Relightful Harmonization: Lighting-aware Portrait Background Replacement (CVPR 2024 · similarity 0.87)
- Painterly Image Harmonization by Learning from Painterly Objects (AAAI 2024 · similarity 0.87)
- FRIH: Fine-Grained Region-Aware Image Harmonization (AAAI 2024 · similarity 0.86)
- Video Harmonization with Triplet Spatio-Temporal Variation Patterns (CVPR 2024 · similarity 0.85)
- Training-and-Prompt-Free General Painterly Harmonization via Zero-Shot Disentenglement on Style and Content References (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
