Research map / Image generation / Editing and style transfer
K-LoRA: Unlocking Training-Free Fusion of Any Subject and Style LoRAs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes K-LoRA, a training-free method that fuses a subject LoRA and a style LoRA per attention layer and per denoising step by comparing Top-K absolute weight sums with a timestep-dependent scaling factor.
Problem
Merging independently trained subject and style LoRAs for diffusion models often loses style details or object identity, and existing fixes either need extra training or manual tuning of ratios and seeds.
Approach
At each attention layer and denoising step, select either the content or style LoRA weights by comparing the sums of their Top-K largest-magnitude elements (K = rank_c × rank_s), with a timestep-dependent scale favoring content early and style late and a global gamma normalizing differences across LoRA sources.
Key results
- K-LoRA reaches a CLIP score of 69.4% and DINO score of 46.9% for subject similarity, above ZipLoRA (64.4%, 35.7%) and B-LoRA (63.8%, 30.6%).
- K-LoRA attains 58.7% style similarity, slightly below ZipLoRA's 60.4% but above B-LoRA's 58.0%.
- In a user study over 22 sets, 52.7% of users preferred K-LoRA, versus 29.2% for ZipLoRA and 18.1% for B-LoRA; GPT-4o preferred K-LoRA 83.3% of the time.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use K-LoRA instead of the strongest baseline for subject preservation | Object-style LoRA fusion · 18 object-style combinations | CLIP Score: 64.4% -> 69.4% | Helped |
| Use K-LoRA instead of the strongest baseline for subject similarity | Object-style LoRA fusion · 18 object-style combinations | DINO Score: 43.0% -> 46.9% | Helped |
| Use K-LoRA instead of the strongest baseline for style similarity | Object-style LoRA fusion · 18 object-style combinations | Style Sim: 68.2% -> 58.7% | Mixed |
| Use K-LoRA rather than ZipLoRA | Object-style LoRA fusion · User study | User Preference: 29.2% -> 52.7% | Helped |
| Use K-LoRA rather than B-LoRA | Object-style LoRA fusion · User study | GPT-4o Feedback: 11.1% -> 83.3% | Helped |
| Remove Top-K selection and use fixed timestep-based selection (ablation) | K-LoRA · SD and FLUX object-style fusion | Visual preservation of subject and style | Hurt |
| Replace Top-K selection with random attention-layer selection (ablation) | K-LoRA · SD and FLUX object-style fusion | Visual preservation of subject and style | Hurt |
| Vary K from small values to larger values (ablation) | K-LoRA · Object-style image generation | Prominence of object and style features | Mixed |
| Use excessively large K (ablation) | K-LoRA · Object-style image generation | Preservation of style and object shape | Hurt |
| Remove the scaling factor (ablation) | K-LoRA · Object-style image generation | Visual generation quality | Hurt |
| Remove gamma from the scaling factor (ablation) | K-LoRA with LoRAs from different sources · Object-style image generation | Style preservation | Hurt |
Surprising findings
- Random assignment of content and style attention layers across steps often retains only one of the object or style features, supporting that the timing of each LoRA matters.
Limitations
- Relies on hyperparameters alpha=1.5 and beta=0.5 and a gamma normalization, tuned for nearly all cases according to authors but not extensively justified.
- Style similarity is slightly lower than ZipLoRA in quantitative results.
- User study and GPT-4o evaluation are on a small set of 22 comparisons.
Experimental setup
Datasets: DreamBooth dataset StyleDrop style images Hugging Face community LoRAs
Benchmarks: DreamBooth subject-style combinations (18 pairs, 10 images each)
Metrics: Style similarity (CLIP) CLIP score DINO score user preference GPT-4o preference
Baselines: Direct merging Joint training B-LoRA ZipLoRA Fixed selection Random selection
Builds on: ZipLoRA B-LoRA Multi-LoRA Composition Magmax DreamBooth
Closest papers by meaning
- NP-LoRA: Null Space Projection for Subject-Style LoRA Fusion (NeurIPS 2026 · similarity 0.90)
- LoRAtorio: An intrinsic approach to LoRA Skill Composition (NeurIPS 2026 · similarity 0.86)
- Merging LoRAs like Playing LEGO: Pushing the Modularity of LoRA to Extremes Through Rank-Wise Clustering (ICLR 2025 · similarity 0.86)
- Cached Multi-Lora Composition for Multi-Concept Image Generation (ICLR 2025 · similarity 0.85)
- Make LoRA Great Again: Boosting LoRA with Adaptive Singular Values and Mixture-of-Experts Optimization Alignment (ICML 2025 · similarity 0.84)
- LoRA Recycle: Unlocking Tuning-Free Few-Shot Adaptability in Visual Foundation Models by Recycling Pre-Tuned LoRAs (CVPR 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.