Research map / Detection and segmentation / Open-set and few-shot detection
Progressive Painterly Image Harmonization from Low-Level Styles to High-Level Styles
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ProPIH, a progressive multi-stage network that harmonizes a photographic foreground on a painterly background from low-level to high-level styles, with a GRU-based early-exit strategy to pick the stage to stop.
Problem
Painterly image harmonization: adjusting the style of a photographic foreground to match an artistic painting background while preserving foreground content; prior feed-forward methods entangle all style levels and act as black boxes.
Approach
A dual-branch network where a frozen-style VGG-19 encoder with AdaIN feeds a bottom branch that harmonizes progressively from low-level to high-level styles, with each stage supervised by style loss up to that level; a GRU predicts the earliest stage at which to exit.
Key results
- ProPIH achieves the highest Bradley-Terry score of 0.917 in the user study, versus 0.474 for the next best baseline PHDNet.
- ProPIH runs in 0.0067 s with 23.45 GFLOPs at 256x256, faster than all listed baselines.
- Annotated exit stages distribution over 5000 training composites: 2.1%, 22.3%, 47.4%, 28.2% for stages 1 to 4.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ProPIH with the predicted exit stage instead of PHDNet | Painterly image harmonization · User study | B-T score: 0.474 -> 0.917 | Helped |
| Use ProPIH with the predicted exit stage instead of E2STN | Painterly image harmonization · Test set, 256 × 256 | Time(s): 0.0079 -> 0.0067 | Helped |
| Use ProPIH with the predicted exit stage instead of E2STN | Painterly image harmonization · Test set, 256 × 256 | FLOPs(G): 29.28 -> 23.45 | Helped |
| Use the full style loss for all four stages instead of stage-specific style losses (ablation) | ProPIH · Four-stage harmonization | harmonization quality | Hurt |
| Remove losses for the first three stages and supervise only the final stage (ablation) | ProPIH · Painterly image harmonization | harmonization quality | Hurt |
| Replace the upsample decoder with bilinear upsampling (ablation) | ProPIH · Painterly image harmonization | harmonization quality | Hurt |
| Reduce the number of fusion blocks in each stage from two to one (ablation) | ProPIH · Painterly image harmonization | harmonization quality | Hurt |
Surprising findings
- Late-stage outputs can contain artifacts or distorted content structure, so exiting early can be better for some composites.
- Even with full style loss at all stages, early-stage outputs remain harmonized only to low style levels.
Limitations
- Relies on manually annotated exit stages for 5000 training images.
- Evaluation relies on a user study with synthetic composites from COCO and WikiArt.
Experimental setup
Datasets: COCO WikiArt
Benchmarks: COCO WikiArt
Metrics: Bradley-Terry score (user study) inference time FLOPs
Baselines: SANet AdaAttN StyTr2 QuantArt INST SDEdit CDC E2STN DPH PHDNet
Builds on: AdaIN VGG-19 GRU curriculum learning
Closest papers by meaning
- Painterly Image Harmonization by Learning from Painterly Objects (AAAI 2024 · similarity 0.92)
- Training-and-Prompt-Free General Painterly Harmonization via Zero-Shot Disentenglement on Style and Content References (AAAI 2025 · similarity 0.90)
- Relightful Harmonization: Lighting-aware Portrait Background Replacement (CVPR 2024 · similarity 0.88)
- FRIH: Fine-Grained Region-Aware Image Harmonization (AAAI 2024 · similarity 0.87)
- High-Resolution Image Harmonization with Adaptive-Interval Color Transformation (NeurIPS 2024 · similarity 0.86)
- ChromaFusionNet (CFNet): Natural Fusion of Fine-Grained Color Editing (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.