Research map / Image generation / Text-to-image diffusion

LoRAtorio: An intrinsic approach to LoRA Skill Composition

NeurIPS 2026 · Image generation › Text-to-image diffusion · approach: Multi-concept composition

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes LoRAtorio, a train-free method that composes multiple LoRA adapters in diffusion models by patch-wise cosine-similarity weighting against the base model's output plus a re-centred classifier-free guidance, extended to dynamic top-k LoRA selection.

Problem

Composing multiple LoRA adapters in text-to-image diffusion models degrades quality, especially in open-ended settings where the needed skills are unknown and LoRAs drift from the base model distribution.

Approach

Weights each LoRA's denoised output per latent patch by SoftMin over cosine similarity to the base model's prediction (LoRAs that diverge more get more weight), and re-centres the unconditional CFG score by mixing in the base model's unconditional output; a top-k variant selects relevant LoRAs at each timestep.

Key results

• On ComposLoRA static setting, LoRAtorio reaches average CLIPScore 36.356 vs 36.199 for the next best (MultLFG), peaking at N=4 with 37.136.
• In the dynamic module selection setting, LoRAtorio keeps average CLIPScore 35.916 while LoRA Merge drops to 27.153.
• GPT-4V evaluation reports up to a 72.43% win rate against prior skill composition methods.

Surprising findings

• Combining LoRAtorio with auto-guidance fails to produce coherent images, which the authors attribute to differing data distributions.
• At N=2 on the ComposLoRA static setting, LoRAtorio's CLIPScore (35.236) is below MultLFG (36.570) and LoraHub (35.681).

Limitations

• Computational cost grows linearly with the number of active LoRAs, since all LoRA outputs are computed in latent space each step.
• Assumes LoRA adapters are well-aligned and semantically coherent; community LoRAs with undocumented training may hurt robustness.
• Failure cases include extra limbs and duplicate characters on SD backbones and concept confusion on Flux.

Experimental setup

Datasets: ComposLoRA

Benchmarks: ComposLoRA

Metrics: CLIPScore GPT-4V pairwise win rate composition quality score image quality score human expert rating

Baselines: Naive LoRA Merge LoRA Switch LoRA Composite LoraHub Switch-A CMLoRA MultLFG

Builds on: LoRA classifier-free guidance ComposLoRA latent diffusion models CMLoRA

Closest papers by meaning

• LoRA-X: Bridging Foundation Models with Training-Free Cross-Model Adaptation (ICLR 2025 · similarity 0.92)
• Cached Multi-Lora Composition for Multi-Concept Image Generation (ICLR 2025 · similarity 0.91)
• SDS-LoRA: Overcoming Anisotropic Gradient Scaling in Low-Rank Adaptation (NeurIPS 2026 · similarity 0.91)
• C-LoRA: Continual Low-Rank Adaptation for Pre-trained Visual Models (NeurIPS 2026 · similarity 0.90)
• Flat-LoRA: Low-Rank Adaptation over a Flat Loss Landscape (ICML 2025 · similarity 0.90)
• The Expressive Power of Low-Rank Adaptation (ICLR 2024 · similarity 0.90)