Research map / Image generation / Text-to-image diffusion
Cached Multi-Lora Composition for Multi-Concept Image Generation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes CMLoRA, a training-free multi-LoRA composition framework that orders LoRAs by Fourier high-frequency profiling and uses non-uniform feature caching to reduce semantic conflicts in multi-concept text-to-image generation.
Problem
Composing multiple LoRAs for multi-concept text-to-image generation degrades image quality as the number of LoRAs grows, due to semantic conflicts between independently trained LoRAs.
Approach
Profiles LoRAs in the Fourier domain by how much high-frequency amplitude they add during denoising, uses this to order and partition LoRAs into high- and low-frequency dominant sets applied at different timesteps, and adds a non-uniform cache interval for non-dominant LoRAs on top of LoRA Composite.
Key results
- On ComposLoRA, CMLoRA (Cache_D) reaches CLIPScore 34.341 at N=5 versus 33.475 for LoRA Switch and 32.582 for LoRA Composite (anime subset averages).
- Abstract reports an average 2.19% CLIPScore gain and 11.25% MLLM win-rate gain over LoraHub, LoRA Composite and LoRA Switch.
- MiniCPM-V average score of CMLoRA (Cache_D) is 8.143 versus 7.110 for the best training-free baseline (LoRA Switch) in the MLLM evaluation table.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use CMLoRA instead of LoraHub | Multi-LoRA image generation · ComposLoRA element-wise average, N=3 | CLIPScore: 35.127 -> 35.699 | Helped |
| Use dynamic caching with CMLoRA (ablation) | CMLoRA · ComposLoRA element-wise average, N=4 | ImageReward: 0.415 -> 0.446 | Helped |
| Use dynamic caching with CMLoRA (ablation) | CMLoRA · ComposLoRA element-wise average, N=5 | ImageReward: 0.151 -> 0.186 | Helped |
| Use dynamic caching with CMLoRA (ablation) | CMLoRA · ComposLoRA element-wise average, N=3 | ImageReward: 0.715 -> 0.700 | Hurt |
| Use dynamic caching with CMLoRA (ablation) | CMLoRA · ComposLoRA element-wise average, N=3 | CLIPScore: 35.699 -> 35.215 | Hurt |
| Use dynamic caching with CMLoRA (ablation) | CMLoRA · ComposLoRA testbed | MiniCPM average score: 8.393 -> 8.143 | Hurt |
| Use CMLoRA instead of Switch | Multi-LoRA image generation · ComposLoRA testbed, N=5 | MACs: 731.811 G -> 1570.335 G | Hurt |
| Use dynamic caching for LoRA Composite (ablation) | LoRA Composite · Multi-concept image generation | Win rate | Helped |
Surprising findings
- LoRA Switch generally outperforms LoRA Composite in CLIPScore, despite Composite injecting all LoRAs at each step.
- CMLoRA incurs substantially higher MACs than Switch (e.g. 1570 G vs 732 G at N=5), a cost trade-off.
Limitations
- CMLoRA has higher computational cost than LoRA Switch and Merge.
- CLIPScore and ImageReward struggle with out-of-distribution anime LoRAs, and the MLLM judge is used to compensate.
- Evaluation is limited to the ComposLoRA testbed with up to 5 LoRAs on Stable Diffusion v1.5.
Experimental setup
Datasets: ComposLoRA
Benchmarks: ComposLoRA
Metrics: CLIPScore ImageReward MiniCPM-V win rate MACs
Baselines: Naive LoRA Merge LoRA Switch LoRA Composite LoraHub Switch-A
Builds on: LoRA DeepCache LoRA Switch LoRA Composite LoraHub
Closest papers by meaning
- LoRAtorio: An intrinsic approach to LoRA Skill Composition (NeurIPS 2026 · similarity 0.91)
- LoRAShop: Training-Free Multi-Concept Image Generation and Editing with Rectified Flow Transformers (NeurIPS 2025 · similarity 0.88)
- Merging LoRAs like Playing LEGO: Pushing the Modularity of LoRA to Extremes Through Rank-Wise Clustering (ICLR 2025 · similarity 0.87)
- NOLA: Compressing LoRA using Linear Combination of Random Basis (ICLR 2024 · similarity 0.87)
- LoRACLR: Contrastive Adaptation for Customization of Diffusion Models (CVPR 2025 · similarity 0.87)
- Flat-LoRA: Low-Rank Adaptation over a Flat Loss Landscape (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.