Research map / Detection and segmentation / CNN features
Painterly Image Harmonization by Learning from Painterly Objects
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Learns a mapping from background style and object features to object style using painterly/photographic object pairs annotated from WikiArt and COCO, and uses the hallucinated style to harmonize composite objects in a feed-forward network.
Problem
Painterly image harmonization: stylizing a photographic object pasted on an artistic painting background so it is compatible with the painterly style, without ground-truth harmonized objects.
Approach
Hallucinate a per-object target style via an ObAdaIN module that maps background style vector and domain-invariant object features to object style, supervised by annotated reference painterly objects that are paired with similar photographic objects.
Key results
- ArtoPIH achieves the highest Bradley-Terry score (0.953) in the user study, versus 0.485 for the next best method PHDNet.
- ArtoPIH runs in 0.0258 s per 256x256 image with 40.03 GFLOPs on an RTX 3090, faster than most baselines except SANet, AdaAttN, and E2STN.
- The dataset construction yields 327,181 triplets from 33,294 painterly objects, averaging 9.83 photographic objects per painterly object.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use ArtoPIH instead of the strongest baseline, PHDNet | Painterly image harmonization · COCO/WikiArt user study | B-T score: 0.485 -> 0.953 | Helped |
| Use ArtoPIH instead of the other feed-forward harmonization methods | Painterly image harmonization · COCO/WikiArt inference (256 × 256) | inference time (s): 0.0079; 0.0258; 0.0321 | Mixed |
Surprising findings
- The harmonized results using hallucinated style are close to those using reference object style, while background-style harmonization differs substantially, showing per-object style differs from global background style.
Limitations
- Unsupervised adversarial domain adaptation was tried and performed unsatisfactorily, so the method relies on costly manual annotation of similar photographic/painterly object pairs.
- Style transfer baselines and local style methods may fail to find relevant regions, while the method's own failure cases are deferred to supplementary material.
Experimental setup
Datasets: WikiArt COCO
Benchmarks: COCO WikiArt
Metrics: Bradley-Terry score from user study inference time FLOPs
Baselines: SANet AdaAttN StyTr2 QuantArt INST SDEdit CDC E2STN DPH PHDNet
Builds on: AdaIN VGG-19 SANet AdaAttN StyTr2
Closest papers by meaning
- Progressive Painterly Image Harmonization from Low-Level Styles to High-Level Styles (AAAI 2024 · similarity 0.92)
- Training-and-Prompt-Free General Painterly Harmonization via Zero-Shot Disentenglement on Style and Content References (AAAI 2025 · similarity 0.88)
- Painterly Image Harmonization in Dual Domains (AAAI 2023 · similarity 0.88)
- Intrinsic Image Harmonization (CVPR 2021 · similarity 0.87)
- Semi-Supervised Parametric Real-World Image Harmonization (CVPR 2023 · similarity 0.86)
- Relightful Harmonization: Lighting-aware Portrait Background Replacement (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.