Research map / Image generation / Text-to-image diffusion
AlignedGen: Aligning Style Across Generated Images
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes AlignedGen, a training-free method that makes attention sharing work in DiT models like Flux for style-consistent image sets, using shifted position embeddings and several attention-sharing refinements.
Problem
Text-to-image diffusion models fail to keep a consistent style across images generated from the same style prompt; existing training-free style-aligned methods (StyleAligned) are built for U-Net and collapse when applied to DiT (e.g., Flux).
Approach
Identifies positional collision from RoPE as the cause of attention-sharing failure in DiT and fixes it with ShiftPE (reference image's positional indices shifted to non-overlapping coordinates); adds Selective Attention Sharing (image-only K/V), key scaling with a factor λ for style strength, and layer-selective application to single blocks; also extracts Q/K/V from external reference images via
Key results
- AlignedGen achieves S_text 0.282, S_sty 0.740, S_dino 0.554, the best on all three metrics versus StyleAligned (0.277/0.728/0.512) and IP-Adapter (0.278/0.737/0.522).
- In a 40-participant user study, AlignedGen ranks first with average rank 1.20 versus 2.04 for IP-Adapter and 3.66 for StyleAligned.
- Applying attention sharing to single blocks of Flux (layers 19-57) with lambda=1.1 gives the best balance of text alignment and style consistency in ablations.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use AlignedGen instead of the strongest reported baseline | Flux.1-dev · StyleAligned test set | S_text: 0.280 -> 0.282 | Helped |
| Use AlignedGen instead of the strongest reported baseline | Flux.1-dev · StyleAligned test set | S_sty: 0.737 -> 0.740 | Helped |
| Use AlignedGen instead of the strongest reported baseline | Flux.1-dev · StyleAligned test set | S_dino: 0.522 -> 0.554 | Helped |
| Use AlignedGen instead of the strongest reported zero-shot editing baseline | Flux.1-dev · Zero-shot editing comparison | S_text: 0.280 -> 0.282 | Helped |
| Use AlignedGen instead of the strongest reported zero-shot editing baseline | Flux.1-dev · Zero-shot editing comparison | S_sty: 0.703 -> 0.740 | Helped |
| Use AlignedGen instead of the strongest reported zero-shot editing baseline | Flux.1-dev · Zero-shot editing comparison | S_dino: 0.487 -> 0.554 | Helped |
| Add Selective Attention Sharing (ablation) | AlignedGen · Ablation study | S_sty: 0.733 -> 0.740 | Helped |
| Add Selective Attention Sharing (ablation) | AlignedGen · Ablation study | S_dino: 0.525 -> 0.554 | Helped |
| Apply AlignedGen to Group 2 (layers 19-38) rather than Group 1 (layers 0-19) (ablation) | AlignedGen layer selection · Ablation study | S_dino: 0.323 -> 0.439 | Helped |
| Apply AlignedGen to Group 1 (layers 0-19) rather than Group 2 (layers 19-38) (ablation) | AlignedGen layer selection · Ablation study | S_sty: 0.697 -> 0.669 | Hurt |
| Apply AlignedGen to all three layer groups rather than Groups 2 and 3 (ablation) | AlignedGen layer selection · Ablation study | S_text: 0.282 -> 0.273 | Hurt |
| Increase lambda from 0.90 to 1.15 (ablation) | AlignedGen key scaling · Ablation study | S_sty: 0.694 -> 0.755 | Helped |
Surprising findings
- Naive attention sharing from StyleAligned collapses on Flux, producing identical outputs that ignore the text prompt, with the root cause traced to RoPE positional collision rather than the sharing mechanism itself.
- Applying attention sharing to the first group of double blocks was ineffective and even detrimental to style consistency.
Limitations
- Evaluation uses only 100 prompt sets from StyleAligned, a small test set.
- Zero-shot editing baselines require meticulous parameter tuning, and the method's own layer range and lambda were selected empirically on this evaluation.
Experimental setup
Datasets: StyleAligned prompt sets (100 prompt sets)
Benchmarks: StyleAligned prompt sets
Metrics: CLIP text alignment (S_text) CLIP style consistency (S_sty) DINO style consistency (S_dino) user study ranking
Baselines: StyleAligned IP-Adapter B-LoRA StyleShot CSGO RF-Solver RF-Edit
Builds on: StyleAligned Flux.1-dev RoPE AdaIN
Closest papers by meaning
- Style Aligned Image Generation via Shared Attention (CVPR 2024 · similarity 0.90)
- Discriminative Probing and Tuning for Text-to-Image Generation (CVPR 2024 · similarity 0.88)
- Aligning Text to Image in Diffusion Models is Easier Than You Think (NeurIPS 2025 · similarity 0.88)
- Towards Transformer-Based Aligned Generation with Self-Coherence Guidance (CVPR 2025 · similarity 0.87)
- CHATS: Combining Human-Aligned Optimization and Test-Time Sampling for Text-to-Image Generation (ICML 2025 · similarity 0.87)
- Direct Consistency Optimization for Robust Customization of Text-to-Image Diffusion models (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.