Research map / Image generation / Editing and style transfer
FRIH: Fine-Grained Region-Aware Image Harmonization
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FRIH, a two-stage global-local image harmonization network that clusters the foreground into adaptive RGB-based submasks and refines a coarse harmonization per region, reaching 38.19 PSNR on iHarmony4 with 11.98M parameters.
Problem
Existing image harmonization methods apply one uniform adjustment to the whole foreground, ignoring that different foreground color regions need different harmonization, which loses specific details.
Approach
A coarse global harmonization followed by adaptive submask extraction (CFSFDP density clustering on pixel RGB values) and a lightweight cascaded U-Net per submask, with a fusion prediction module that combines features from all cascaded decoder layers.
Key results
- FRIH achieves 38.19 dB PSNR and 23.98 MSE on the iHarmony4 test set, above the next best Harmonizer at 37.84 dB.
- Ablation: adding submask-based cascaded module raises PSNR over the baseline by 1.08 dB; adding fusion prediction adds a further 0.29 dB.
- FRIH has 11.98M parameters, fewer than DoveNet (54.76M), IIH (40.83M), and RainNet (54.75M).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FRIH instead of Harmonizer | Image harmonization · iHarmony4 test | PSNR: 37.84 -> 38.19 | Helped |
| Use FRIH instead of Harmonizer | Image harmonization · iHarmony4 test | MSE: 24.26 -> 23.98 | Helped |
| Add the cascaded module using the global mask instead of only the base network (ablation) | FRIH · iHarmony4 test | PSNR: 36.82 -> 36.98 | Helped |
| Add the submask-based cascaded module instead of using only the base network (ablation) | FRIH · iHarmony4 test | PSNR: 36.82 -> 37.90 | Helped |
| Use extracted submasks instead of the global mask in the cascaded module (ablation) | FRIH · iHarmony4 test | PSNR: 36.98 -> 37.90 | Helped |
| Add fusion prediction to the submask-based cascaded method (ablation) | FRIH · iHarmony4 test | PSNR: 37.90 -> 38.19 | Helped |
| Set submask extraction cutoff distance dc to 0.1 rather than 0.01 (ablation) | FRIH · iHarmony4 test | PSNR: 38.09 -> 38.19 | Helped |
| Set submask extraction cutoff distance dc to 0.1 rather than 0.05 (ablation) | FRIH · iHarmony4 test | PSNR: 38.17 -> 38.19 | Helped |
| Set submask extraction cutoff distance dc to 0.2 rather than 0.1 (ablation) | FRIH · iHarmony4 test | PSNR: 38.19 -> 38.15 | Hurt |
| Set submask extraction cutoff distance dc to 0.3 rather than 0.1 (ablation) | FRIH · iHarmony4 test | PSNR: 38.19 -> 38.08 | Hurt |
| Set submask extraction cutoff distance dc to 0.4 rather than 0.1 (ablation) | FRIH · iHarmony4 test | PSNR: 38.19 -> 38.02 | Hurt |
Surprising findings
- Using the global mask instead of submasks in the cascaded module gives only a 0.16 dB gain over baseline, suggesting the submask decomposition drives most of the improvement.
- The method gives little benefit on the Hday2night subset, where foregrounds are homogeneous (sky, water), making sub-region separation hard.
Limitations
- Submask count is capped at 10 based on the authors' observation that suitable counts rarely exceed 10.
- Performance depends on the cutoff distance dc, which is hand-set to 0.1 with a sensitivity sweep.
- Little gain on homogeneous foregrounds such as sky or water on Hday2night.
Experimental setup
Datasets: iHarmony4
Benchmarks: iHarmony4
Metrics: MSE PSNR fMSE LPIPS
Baselines: DIH S2AM DoveNet BargainNet IIH RainNet iDIH D-HT S2CRNET Harmonizer
Builds on: U-Net CFSFDP clustering DoveNet RainNet
Closest papers by meaning
- Progressive Painterly Image Harmonization from Low-Level Styles to High-Level Styles (AAAI 2024 · similarity 0.87)
- Training-and-Prompt-Free General Painterly Harmonization via Zero-Shot Disentenglement on Style and Content References (AAAI 2025 · similarity 0.86)
- Video Harmonization with Triplet Spatio-Temporal Variation Patterns (CVPR 2024 · similarity 0.84)
- High-Resolution Image Harmonization with Adaptive-Interval Color Transformation (NeurIPS 2024 · similarity 0.84)
- Relightful Harmonization: Lighting-aware Portrait Background Replacement (CVPR 2024 · similarity 0.84)
- Prior-guided Hierarchical Harmonization Network for Efficient Image Dehazing (AAAI 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.