Research map / Image generation / Editing and style transfer
SaMam: Style-aware State Space Model for Arbitrary Image Style Transfer
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes SaMam, a Mamba-based encoder-decoder for arbitrary image style transfer with a style-aware VSS decoder, zigzag scanning, and local enhancement, achieving better quality and efficiency than CNN, Transformer, and diffusion baselines.
Problem
Image style transfer needs a global effective receptive field, but CNNs, Transformers and diffusion models get it only at high computational cost; SSMs offer linear-complexity long-range modeling but are not designed for style-aware, spatially continuous 2D processing.
Approach
Adapts Mamba to arbitrary style transfer with a style-aware S6 (S7) block whose A and D SSM parameters are predicted from style embeddings, style-aware convolution/instance norm/channel modulation, a zigzag 2D scan for spatial continuity, and a local enhancement module against local pixel forgetting and channel redundancy.
Key results
- SaMam reports the best ArtFID (26.305 vs 29.837 for AesPA), FID (17.946), LPIPS (0.3884) and CFSD (0.2703) among compared methods in Table 1.
- SaMam runs at 0.034 s and 77.1 G MACs at 512x512, second only to the CNN method EFDM in efficiency.
- Ablations show removing the LoE raises ArtFID from 26.305 to 28.607, and replacing SConv with DWConv raises it to 31.446.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use SaMam instead of AesPA | Arbitrary style transfer · 800 stylized images | ArtFID: 29.837 -> 26.305 | Helped |
| Use SaMam instead of AesPA | Arbitrary style transfer · 800 stylized images | LPIPS: 0.4050 -> 0.3884 | Helped |
| Use SaMam instead of AesPA | Arbitrary style transfer · 800 stylized images | FID: 20.236 -> 17.946 | Helped |
| Use SaMam instead of CAPVST | Arbitrary style transfer · 800 stylized images | CFSD: 0.2912 -> 0.2703 | Helped |
| Use SaMam instead of EFDM | 512 × 512 output resolution · 512 × 512 output resolution | Time (s): 0.027 -> 0.034 | Hurt |
| Use SaMam instead of EFDM | 512 × 512 output resolution · 512 × 512 output resolution | MACs (G): 63.3 -> 77.1 | Hurt |
| Add zigzag scan instead of cross scan (ablation) | SaMam · Ablation study | ArtFID: 26.808 -> 26.305 | Helped |
| Add local enhancement (LoE) (ablation) | SaMam · Ablation study | ArtFID: 28.607 -> 26.305 | Helped |
| Use style-aware S7 block instead of S6 block (ablation) | SaMam · Ablation study | ArtFID: 30.476 -> 26.305 | Helped |
| Use style-aware convolution (SConv) instead of DWConv (ablation) | SaMam · Ablation study | ArtFID: 31.446 -> 26.305 | Helped |
| Add style-aware channel modulation (SCM) (ablation) | SaMam · Ablation study | CSFD: 0.3235 -> 0.2703 | Helped |
| Use zero-initialized SAIN instead of instance norm (ablation) | SaMam · Ablation study | ArtFID: 30.112 -> 26.305 | Helped |
Surprising findings
- Diffusion-based methods such as ZStar and StyleID scored worst on content fidelity, with CFSD values of 0.7294 and 0.4277 respectively against 0.2703 for SaMam.
Limitations
- Trained only on MS-COCO and WikiArt at 256x256 resolution, with evaluation on a limited set of 20 content and 40 style images.
- Comparisons rely on official code with default configurations, and the reported 0.034 s runtime is for a single RTX 3090.
Experimental setup
Datasets: MS-COCO WikiArt
Benchmarks: MS-COCO content / WikiArt style test pairs (20 content x 40 style images)
Metrics: LPIPS FID ArtFID CFSD MACs inference time parameters
Baselines: AesPA EFDM ATK UCAST StyTr2 S2WAT StyleFormer STTR ArtFlow CAPVST DiffuseIT ZStar
Builds on: VMamba Vision Mamba Mamba AdaConv StyleID
Closest papers by meaning
- MicroAST: Towards Super-fast Ultra-Resolution Arbitrary Style Transfer (AAAI 2023 · similarity 0.87)
- StyleBank: An Explicit Representation for Neural Image Style Transfer (CVPR 2017 · similarity 0.87)
- Learning Linear Transformations for Fast Image and Video Style Transfer (CVPR 2019 · similarity 0.86)
- SCSA: A Plug-and-Play Semantic Continuous-Sparse Attention for Arbitrary Semantic Style Transfer (CVPR 2025 · similarity 0.86)
- A Content Transformation Block for Image Style Transfer (CVPR 2019 · similarity 0.86)
- ArtBank: Artistic Style Transfer with Pre-trained Diffusion Model and Implicit Style Prompt Bank (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.