Research map / Image generation / Editing and style transfer
AesFA: An Aesthetic Feature-Aware Arbitrary Neural Style Transfer
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AesFA, a lightweight end-to-end arbitrary style transfer model that uses octave convolutions for frequency decomposition, an AdaOct stylization module, and a k-nearest-negative aesthetic contrastive loss, running under 0.02 s per image without pre-trained networks at inference.
Problem
Existing neural style transfer methods either fail to transfer aesthetic style information (tones, brushstrokes, textures, grains, local structure) or are computationally expensive due to pre-trained VGG feature extractors and inefficient contrastive training, which limits high-resolution and real-t
Approach
Decompose content and style features into high- and low-frequency branches using octave convolutions, modulate content with predicted aesthetic-aware kernels and biases (AdaOct, an AdaConv followed by OctConv), and train with a contrastive loss that uses only the k-th nearest negatives (k=1) computed via EFDM on VGG features, with VGG used only during training.
Key results
- At 256^2, AesFA achieves 32.90% user preference and 0.016 s inference, with LPIPS 0.368 (best).
- At 4K, AesFA achieves the lowest style loss (0.216) and 50.60% user preference, while competitors such as AdaAttn, AesUST, IECAST and StyTr^2 run out of GPU memory.
- AesFA infers a single image in under 0.02 seconds across resolutions from 256 to 4K.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add aesthetic feature contrastive loss (ablation) | AesFA · Stylization results (ablation study) | Stylization quality | Helped |
| use OctConv with α=0.5 (ablation) | AesFA · Stylization results (ablation study) | Stylization quality | Helped |
| replace standard convolutions with OctConv (ablation) | AesFA · Stylization results (ablation study) | Stylization quality | Helped |
| select the nearest style image (k=1) for contrastive negatives (ablation) | AesFA aesthetic feature contrastive loss · Aesthetic feature contrastive learning | Performance | Helped |
Surprising findings
- StyTr^2 obtains the best style loss at 256^2 (0.581) and AdaAttn the best SSIM there, so AesFA is not best on every metric at every resolution.
- MicroAST is fastest at 256^2 and 1024^2 in inference time, slightly ahead of AesFA.
Limitations
- Comparisons rely on retraining authors' released code with default configurations, which may not reflect each method's best settings.
- Quantitative metrics do not consistently favor AesFA; the user study is subjective with 79 participants.
Experimental setup
Datasets: COCO WikiArt
Benchmarks: WikiArt pexels.com images (ultra-high resolution)
Metrics: style loss (VGG space) LPIPS SSIM inference time user preference
Baselines: AdaConv AdaIN MicroAST EFDM AdaAttn AesUST IECAST StyTr^2
Builds on: AdaConv MicroAST OctConv EFDM MobileNet
Closest papers by meaning
- Style-Aware Normalized Loss for Improving Arbitrary Style Transfer (CVPR 2021 · similarity 0.90)
- MetaStyle: Three-Way Trade-off among Speed, Flexibility, and Quality in Neural Style Transfer (AAAI 2019 · similarity 0.88)
- Universal Style Transfer via Feature Transforms (NeurIPS 2017 · similarity 0.87)
- Arbitrary Style Transfer With Deep Feature Reshuffle (CVPR 2018 · similarity 0.87)
- OmniStyle: Filtering High Quality Style Transfer Data at Scale (CVPR 2025 · similarity 0.86)
- ELTA: An Enhancer against Long-Tail for Aesthetics-oriented Models (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.