Research map / Image generation / Text-to-image diffusion
Style Aligned Image Generation via Shared Attention
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces StyleAligned, a training-free method that shares self-attention from a reference image (with AdaIN on queries and keys) across a batch of SDXL generations to produce style-consistent image sets.
Problem
Text-to-image diffusion models produce inconsistent styles across a set of images generated from the same style description, and existing style personalization requires per-style fine-tuning and manual data curation.
Approach
Share attention from all target images to only the first (reference) image in the batch, normalizing target queries and keys with AdaIN toward the reference statistics, so the style propagates without any optimization or fine-tuning; DDIM inversion extends this to reference images.
Key results
- Achieves much higher set style consistency (DINO similarity) than SDXL reference, at the cost of some text alignment, per the Fig. 6 chart.
- In a user study with 2400 answers from 100 MTurk users, StyleAligned was preferred over StyleDrop (unofficial MUSE) 85.2%, StyleDrop (SDXL) 67.1%, and DreamBooth-LoRA (SDXL) 61.3%.
- Generating a four-image set takes 29 seconds on a single A100 GPU with shared attention across all 70 self-attention layers of SDXL.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use StyleAligned instead of StyleDrop (unofficial MUSE) | Style-aligned image set generation · User study | judgments in favor of StyleAligned: 85.2 % | Helped |
| Use StyleAligned instead of StyleDrop (SDXL) | Style-aligned image set generation · User study | judgments in favor of StyleAligned: 67.1 % | Helped |
| Use StyleAligned instead of DreamBooth-LoRA (SDXL) | Style-aligned image set generation · User study | judgments in favor of StyleAligned: 61.3% | Helped |
| Share attention across all images instead of sharing only with the reference image (ablation) | StyleAligned · Style-aligned image sets | style consistency | Helped |
| Share attention across all images instead of sharing only with the reference image (ablation) | StyleAligned · Style-aligned image sets | text alignment | Hurt |
| Share attention across all images instead of sharing only with the reference image (ablation) | StyleAligned · Style-aligned image sets | image-set quality and diversity | Hurt |
| Remove AdaIN normalization of target queries and keys (ablation) | StyleAligned · Style-aligned image sets | style consistency | Hurt |
| Reduce the number of shared attention layers (ablation) | StyleAligned · Style-aligned image sets | image-set diversity | Helped |
| Reduce the number of shared attention layers (ablation) | StyleAligned · Style-aligned image sets | style alignment | Hurt |
Surprising findings
- Full attention sharing across all images gives higher style consistency but causes content leakage between images and reduces diversity across sets.
- Encoder-based personalization methods (ELITE, IP-Adapter, BLIP-Diffusion) perform poorly on style-aligned generation because they struggle to separate style from content.
Limitations
- Trades text alignment for style consistency relative to the unmodified SDXL reference.
- DDIM inversion for real input images may fail or produce erroneous trajectories.
- Encoder-based personalization baselines and style-aligned generation rely on a single reference, limiting control over shape and appearance similarity.
Experimental setup
Benchmarks: 100-prompt style evaluation set (ChatGPT-generated)
Metrics: CLIP score (text alignment) DINO ViT-B/8 pairwise cosine similarity (set consistency) user preference rate
Baselines: StyleDrop (unofficial MUSE) StyleDrop (SDXL) DreamBooth-LoRA (SDXL) IP-Adapter ELITE BLIP-Diffusion Full attention sharing ablation Without AdaIN ablation
Builds on: Stable Diffusion XL prompt-to-prompt AdaIN DDIM inversion DreamBooth
Closest papers by meaning
- AlignedGen: Aligning Style Across Generated Images (NeurIPS 2025 · similarity 0.90)
- Direct Consistency Optimization for Robust Customization of Text-to-Image Diffusion models (NeurIPS 2024 · similarity 0.89)
- StyleStudio: Text-Driven Style Transfer with Selective Control of Style Elements (CVPR 2025 · similarity 0.89)
- Discriminative Probing and Tuning for Text-to-Image Generation (CVPR 2024 · similarity 0.88)
- OneIG-Bench: Omni-dimensional Nuanced Evaluation for Image Generation (NeurIPS 2025 · similarity 0.88)
- TextCenGen: Attention-Guided Text-Centric Background Adaptation for Text-to-Image Generation (ICML 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.