Research map / Image generation / Editing and style transfer
Puff-Net: Efficient Style Transfer with Pure Content and Style Feature Fusion Network
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Puff-Net, an encoder-only transformer for arbitrary style transfer that operates on purified content and style features extracted by an INN-based content extractor and an LT-block style extractor.
Problem
Arbitrary style transfer with transformers is expensive (high memory and slow inference), and existing methods often under-stylize or lose content structure.
Approach
Replace the full encoder-decoder transformer with an encoder-only design where content tokens form the query and style tokens form key/value, plus separate feature extractors (INN for content, LT block for style) that purify inputs before fusion.
Key results
- Inference time at 256x256 is 0.098 s for Puff-Net versus 0.116 s for StyTr2 and 0.661 s at 512x512 for StyTr2 versus 0.134 s for Puff-Net.
- Content loss Lc of 1.92 for Puff-Net vs 1.89 for StyTr2 and 0.86 for CAP-VSTNet; style loss Ls of 2.21 for Puff-Net vs 1.69 for StyTr2.
- User study with 55 participants found Puff-Net's stylization ability best, with content preservation similar to CAP-VSTNet.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use content and style feature extractors before the encoder-based transformer (ablation) | Puff-Net · Style transfer | visual content preservation and stylization | Helped |
| Use CAPE instead of traditional sinusoidal positional encoding (ablation) | Puff-Net · Style transfer | visual stylization quality | Helped |
| Initialize output embedding εo with content embedding εc rather than style embedding εs (ablation) | Puff-Net · Style transfer | content-consistent stylization | Helped |
| Initialize output embedding εo randomly (ablation) | Puff-Net · Style transfer | generation quality | Hurt |
| Initialize output embedding εo with zeros (ablation) | Puff-Net · Style transfer | generation quality | Hurt |
| Freeze the style extractor after 12,000 training iterations | Puff-Net · Style transfer | image style | Helped |
| Use Puff-Net instead of StyTr2 | Style transfer · 256 × 256 | inference time (seconds): 0.116 -> 0.098 | Helped |
| Use Puff-Net instead of StyTr2 | Style transfer · 512 × 512 | inference time (seconds): 0.661 -> 0.134 | Helped |
| Use Puff-Net instead of StyTr2 | Style transfer · 200 stylized images | content perceptual loss (Lc): 1.89 -> 1.92 | Hurt |
| Use Puff-Net instead of StyTr2 | Style transfer · 200 stylized images | style perceptual loss (Ls): 1.69 -> 2.21 | Hurt |
Surprising findings
- Initializing the output embedding with style tokens or using random/zero initialization yields stylized images that are very similar to the style image or blank, so content initialization is required.
- Style feature extractor was frozen after 12,000 iterations because longer training caused style features to disappear as content loss dominated.
Limitations
- Unreasonable stylization when content and style images are complex.
- Style difference loss is slightly higher than some baselines, and content loss is behind CAP-VSTNet.
Experimental setup
Datasets: MS-COCO WikiArt
Benchmarks: MS-COCO WikiArt
Metrics: content perceptual loss style perceptual loss inference time user study preference
Baselines: CAP-VSTNet StyTr2 StyleFormer IEST AdaIN SANet MAST ArtFlow AdaAttN
Builds on: StyTr2 ViT AdaIN MobileNetV2 bottleneck residual block LT block
Closest papers by meaning
- Master: Meta Style Transformer for Controllable Zero-Shot and Few-Shot Artistic Style Transfer (CVPR 2023 · similarity 0.89)
- Separating Style and Content for Generalized Style Transfer (CVPR 2018 · similarity 0.88)
- Neural Style Transfer via Meta Networks (CVPR 2018 · similarity 0.88)
- Multimodal Transfer: A Hierarchical Deep Convolutional Neural Network for Fast Artistic Style Transfer (CVPR 2017 · similarity 0.88)
- OmniStyle: Filtering High Quality Style Transfer Data at Scale (CVPR 2025 · similarity 0.87)
- HSI: A Holistic Style Injector for Arbitrary Style Transfer (CVPR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.