Research map / Architectures and efficiency / Attention and transformers
S2WAT: Image Style Transfer via Hierarchical Vision Transformer Using Strips Window Attention
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes S2WAT, a hierarchical vision transformer for arbitrary image style transfer that uses strip and square window attention fused by an "Attn Merge" similarity-weighted fusion to address the grid-like locality problem.
Problem
Window-based transformers used for style transfer produce grid-like or block artifacts due to the locality of window attention, while global attention loses local detail.
Approach
SpW Attention runs horizontal-strip, vertical-strip and square window attentions in parallel and fuses them with "Attn Merge", which weights each window output by its similarity to the input, instead of static sum or concatenation.
Key results
- S2WAT achieves the lowest content loss (1.66) and identity losses (0.16, 1.38) and the highest SSIM (0.651) among compared methods, while SANet has the lowest style loss (1.11) and S2WAT's style loss is 1.74.
- In a user study with 3002 votes from 72 participants, S2WAT received 25.4% of votes, the highest among compared methods (StyTr2 23.6%).
- Replacing Attn Merge with sum produces heavily corrupted outputs, and concatenation loses much input information, per the qualitative ablation.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace the S2WAT encoder with a Swin-based encoder (ablation) | S2WAT · MS-COCO/WikiArt style transfer | SSIM: 0.651 -> 0.567 | Hurt |
| Reduce strip widths from 2, 4, 7 to 1, 1, 1 (ablation) | S2WAT · MS-COCO/WikiArt style transfer | SSIM: 0.651 -> 0.647 | Hurt |
| Use only strip-based attentions (ablation) | S2WAT · MS-COCO/WikiArt style transfer | Style Loss: 1.74 -> 2.00 | Hurt |
| Replace Attn Merge with sum fusion (ablation) | S2WAT SpW Attention · Image style transfer | Stylized image quality | Hurt |
| Replace Attn Merge with concatenation (ablation) | S2WAT SpW Attention · Image style transfer | Input information retention | Hurt |
| Use horizontal strip attention alone (ablation) | S2WAT SpW Attention · Image style transfer | Stylized image quality | Hurt |
| Use vertical strip attention alone (ablation) | S2WAT SpW Attention · Image style transfer | Stylized image quality | Hurt |
| Use square window attention alone (ablation) | S2WAT SpW Attention · Image style transfer | Stylized image quality | Hurt |
Surprising findings
- Mix-FFN convolutions added to a Swin encoder do not eliminate blocky artifacts.
- Simple sum fusion of window attention outputs causes optimization failure and corrupted stylized images.
Limitations
- Cannot produce completely content-unbiased results; content leak persists under repeated stylization.
- SpW Attention complexity is not linear in patch number, limiting scalability.
- Evaluation relies on losses as indirect metrics and a user study rather than standard style transfer benchmarks.
Experimental setup
Datasets: MS-COCO WikiArt
Benchmarks: MS-COCO WikiArt
Metrics: content loss style loss identity loss 1 identity loss 2 SSIM user preference percentage
Baselines: AdaIN WCT SANet MCC ArtFlow IEST CAST StyTr2 InST
Builds on: Swin Transformer StyTr2 CSWin AdaIN
Closest papers by meaning
- HSI: A Holistic Style Injector for Arbitrary Style Transfer (CVPR 2025 · similarity 0.85)
- Puff-Net: Efficient Style Transfer with Pure Content and Style Feature Fusion Network (CVPR 2024 · similarity 0.85)
- Attention Distillation: A Unified Approach to Visual Characteristics Transfer (CVPR 2025 · similarity 0.83)
- SaMam: Style-aware State Space Model for Arbitrary Image Style Transfer (CVPR 2025 · similarity 0.83)
- OmniStyle: Filtering High Quality Style Transfer Data at Scale (CVPR 2025 · similarity 0.83)
- SCSA: A Plug-and-Play Semantic Continuous-Sparse Attention for Arbitrary Semantic Style Transfer (CVPR 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.