Research map / Image generation / Editing and style transfer
StyleStudio: Text-Driven Style Transfer with Selective Control of Style Elements
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes StyleStudio, a fine-tuning-free text-driven style transfer method built on CSGO that adds cross-modal AdaIN fusion, style-based classifier-free guidance with a ControlNet-generated negative style image, and a teacher model sharing self-attention maps early in denoising.
Problem
Text-driven style transfer suffers from style overfitting to the reference image (dominant colors/patterns override the text), limited selective control over style elements, and layout instability/artifacts such as checkerboard patterns.
Approach
Replace additive text-image conditioning with a cross-modal AdaIN that normalizes text cross-attention features by style statistics; extend classifier-free guidance to style by using a ControlNet-generated negative style image as the unconditional counterpart; and share self-attention maps from an unstylized teacher SDXL model during early denoising steps to stabilize layout.
Key results
- Text alignment score 0.216 for the proposed method versus 0.180 to 0.235 for baselines in Table 1 (the table reports a metric where the ranking is not clearly 'lower is better' in the text; authors claim best text alignment)
- Ablation: cross-modal AdaIN alone improves text alignment by 3.2% to 0.223 and teacher model alone to 0.228 (5.5%); combined reach 0.216 as stated in the ablation table
- User study with 49 participants: 62.92% text-alignment votes and 50.85% style-similarity votes for the proposed method, the highest among compared methods
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add Cross-Modal AdaIN and the Teacher Model (ablation) | CSGO · 52 prompts and 20 style reference images | text alignment: 0.216 -> 0.235 | Helped |
| Add the Teacher Model without Cross-Modal AdaIN (ablation) | CSGO · 52 prompts and 20 style reference images | text alignment: 0.216 -> 0.223 (+3.2%) | Helped |
| Add Cross-Modal AdaIN without the Teacher Model (ablation) | CSGO · 52 prompts and 20 style reference images | text alignment: 0.216 -> 0.228 (+5.5%) | Helped |
| Apply negative-style image guidance to suppress unwanted style elements (ablation) | StyleStudio SCFG · Style transfer with mixed style elements | control of unintended style elements | Helped |
| Use a negative text prompt to suppress unwanted style elements (ablation) | StyleStudio · Style transfer with mixed style elements | control of unintended style elements | Mixed |
| Stop Teacher Model involvement too early (ablation) | StyleStudio · Style transfer layout stability | layout stability | Hurt |
| Use the Teacher Model throughout the full denoising process (ablation) | StyleStudio · Style transfer style preservation | style preservation | Hurt |
Surprising findings
- The paper reports that a teacher model's involvement for the full denoising process (e.g., up to timestep 50) noticeably dilutes style, so it must be stopped early
Limitations
- Teacher model slightly increases inference time
- Generating the negative style image requires expertise and manual effort
- Ablation and key numbers are presented inconsistently in the text
Experimental setup
Datasets: StyleShot style images StyleAdapter prompts
Benchmarks: 52-prompt by 20-style benchmark 30 ChatGPT-generated color-object prompts
Metrics: CLIP text alignment (cosine similarity, lower is better as r user study text alignment vote rate user study style similarity vote rate inference time
Baselines: IP-Adapter InstantStyle CSGO StyleAlign StyleCrafter StyleShot DEADiff
Builds on: CSGO classifier-free guidance AdaIN IP-Adapter
Closest papers by meaning
- Style Aligned Image Generation via Shared Attention (CVPR 2024 · similarity 0.89)
- Text to Sketch Generation with Multi-Styles (NeurIPS 2025 · similarity 0.88)
- FineStyle: Fine-grained Controllable Style Personalization for Text-to-image Models (NeurIPS 2024 · similarity 0.88)
- CSGO: Content-Style Composition in Text-to-Image Generation (NeurIPS 2025 · similarity 0.88)
- SigStyle: Signature Style Transfer via Personalized Text-to-Image Models (AAAI 2025 · similarity 0.88)
- ArtAdapter: Text-to-Image Style Transfer using Multi-Level Style Encoder and Explicit Adaptation (CVPR 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.