Research map / Architectures and efficiency / Pruning and efficient tuning
QR-LoRA: Efficient and Disentangled Fine-tuning via QR Decomposition for Customized Generation
Jiahui Yang, Yongjia Ma, Donglin Di, Jianxun Cui, Hao Li, Wei Chen, Yan Xie, Xun Yang, Wangmeng Zuo
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| train only the delta-R matrix with fixed Q and R instead of training full LoRA A and B (ablation) | QR-LoRA · content-style fusion | cosine similarity between task updates: max cosine similarity between delta-R matrices below 0.2 | Improved |
| directly fine-tune Q and R instead of only delta-R (ablation) | QR-LoRA · content-style fusion | feature disentanglement (cosine similarity): R matrix similarity > 0.75 | Hurt |
| replace B-LoRA / ZipLoRA with QR-LoRA on SDXL | SDXL · content-style fusion | user study rating (0-5): 3.34 -> 4.07 (vs B-LoRA) | Improved |
| replace direct merge DB-LoRA with QR-LoRA on SD3 | SD3 · content-style fusion | user study rating (0-5): 2.93 -> 4.14 | Improved |
| replace direct merge DB-LoRA with QR-LoRA on FLUX.1-dev | FLUX.1-dev · content-style fusion | DINO-C: 0.654 -> 0.687 | Improved |
| vary merge scaling coefficients lambda_c and lambda_s (ablation) | QR-LoRA · content-style fusion | generation quality | No clear effect |
| train only delta-R versus standard LoRA (ablation) | QR-LoRA · convergence | convergence speed | No clear effect |
| train both Q and R instead of only delta-R (ablation) | QR-LoRA · convergence | convergence speed | Improved |
| use QR decomposition-based delta-R reduces trainable parameters | QR-LoRA · parameter count | trainable parameters: half of conventional LoRA | Improved |
Closest papers by meaning
- DEFT: Decompositional Efficient Fine-Tuning for Text-to-Image Models (NeurIPS 2025 · similarity 0.88)
- NP-LoRA: Null Space Projection for Subject-Style LoRA Fusion (NeurIPS 2026 · similarity 0.87)
- GraLoRA: Granular Low-Rank Adaptation for Parameter-Efficient Fine-Tuning (NeurIPS 2025 · similarity 0.86)
- LoRAtorio: An intrinsic approach to LoRA Skill Composition (NeurIPS 2026 · similarity 0.86)
- Navigating Text-To-Image Customization: From LyCORIS Fine-Tuning to Model Evaluation (ICLR 2024 · similarity 0.86)
- Cached Multi-Lora Composition for Multi-Concept Image Generation (ICLR 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
