Research map / Image generation / Editing and style transfer
HSI: A Holistic Style Injector for Arbitrary Style Transfer
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Holistic Style Injector (HSI), a linear-complexity attention-like style transfer module that uses global style statistics and element-wise multiplication with dynamic dual relations, plugged into a VGG-19 encoder-decoder for arbitrary style transfer.
Problem
Attention-based arbitrary style transfer relies on point-wise matching that overfocuses on local style patterns and has quadratic complexity, causing disharmonious patterns and memory overflow on high-resolution images.
Approach
Replace point-wise attention similarity with global channel-wise style statistics (mean, variance, skewness, kurtosis) and compute content-style relations via element-wise multiplication, weighted by a dynamic dual relation (local-content-to-global-style and global-content-to-global-style) controlled by a cosine-similarity coefficient, giving O(HWC) complexity.
Key results
- Achieves lowest style loss (0.95), LPIPS (0.46) and FID (18.46) among compared AST methods on 400 stylized images, with content loss 0.62 second only to StyA2K's 0.59.
- Runs at 2048x2048 resolution on a 24GB GPU where SANet, AdaAttN, StyTr2, StyA2K and AesPA-Net run out of memory; uses about 2GB and roughly 50 fps at 1K resolution.
- Plugging HSI into SANet in place of self-attention reduces repetitive and distorted patterns (qualitative only).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use HSI instead of AesPA-Net | AST · AST evaluation (400 stylized images) | Style Loss: 0.99 -> 0.95 | Helped |
| use HSI instead of StyA2K | AST · AST evaluation (400 stylized images) | LPIPS: 0.49 -> 0.46 | Helped |
| use HSI instead of SANet | AST · AST evaluation (400 stylized images) | FID: 18.74 -> 18.46 | Helped |
| use HSI instead of StyA2K | AST · AST evaluation (400 stylized images) | Content Loss: 0.59 -> 0.62 | Hurt |
| add mean style statistic (ablation) | HSI · HSI stylization examples | qualitative style rendering | Helped |
| add variance style statistic (ablation) | HSI · HSI stylization examples | qualitative style rendering | Helped |
| add kurtosis style statistic (ablation) | HSI · HSI stylization examples | qualitative style rendering | Helped |
| add skewness style statistic (ablation) | HSI · HSI stylization examples | qualitative style rendering | Helped |
| add global-content-to-global-style relation to the local-content-to-global-style relation (ablation) | HSI · HSI stylization examples | content-style consistency | Helped |
| add local-content-to-global-style relation to the global-content-to-global-style relation (ablation) | HSI · HSI stylization examples | content-style consistency | Helped |
| replace SANet self-attention with HSI (ablation) | SANet · SANet stylization examples | qualitative stylization quality | Helped |
Limitations
- Quantitative evaluation uses only 400 stylized images from 20 content and 20 style images.
- Content loss is slightly worse than StyA2K.
- Evaluation of generalization is qualitative only (SANet + HSI).
Experimental setup
Datasets: MS-COCO WikiArt
Benchmarks: MS-COCO WikiArt
Metrics: content loss style loss LPIPS FID inference time GPU memory
Baselines: AdaIN SANet AdaAttN StyTr2 StyA2K AesPA-Net MicroAST
Builds on: SANet self-attention AdaIN dynamic network
Closest papers by meaning
- SCSA: A Plug-and-Play Semantic Continuous-Sparse Attention for Arbitrary Semantic Style Transfer (CVPR 2025 · similarity 0.89)
- OmniStyle: Filtering High Quality Style Transfer Data at Scale (CVPR 2025 · similarity 0.89)
- Style Injection in Diffusion: A Training-free Approach for Adapting Large-scale Diffusion Models for Style Transfer (CVPR 2024 · similarity 0.88)
- Z*: Zero-shot Style Transfer via Attention Reweighting (CVPR 2024 · similarity 0.87)
- ArtBank: Artistic Style Transfer with Pre-trained Diffusion Model and Implicit Style Prompt Bank (AAAI 2024 · similarity 0.87)
- ArtAdapter: Text-to-Image Style Transfer using Multi-Level Style Encoder and Explicit Adaptation (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.