Research map / Architectures and efficiency / Pruning and efficient tuning

NP-LoRA: Null Space Projection for Subject-Style LoRA Fusion

NeurIPS 2026 · Architectures and efficiency › Pruning and efficient tuning · approach: LoRA variants and theory · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes NP-LoRA, a training-free method that fuses content and style LoRAs by projecting the content LoRA onto the null space of the style LoRA's right singular subspace, with a closed-form soft projection controlled by a single parameter mu.

Problem

Composing independently trained subject (content) and style LoRAs for diffusion models without retraining often degrades either subject identity or style, since their weight updates interfere along shared directions.

Approach

Treat LoRA fusion as controlling interactions in overlapping low-rank subspaces: project the content LoRA onto the complement of the style LoRA's dominant right singular subspace, and relax this hard projection into a regularized closed-form soft projection (I + mu P)^{-1} that interpolates between linear merging and hard projection.</ntext>

Key results

• NP-LoRA reaches S_arith 0.55 and S_harm 0.50 on 32 SDXL content-style pairs, versus 0.53/0.46 for the best baseline (DuoLoRA).
• In a 20-person user study NP-LoRA was preferred in 46.88% of cases, and GPT-5 preferred it in 48.59%, versus at most 15.63% for any baseline.
• On community FLUX LoRAs NP-LoRA improves S_harm to 0.54 from 0.50 for K-LoRA.

Surprising findings

• Projecting the style LoRA into the content null space substantially weakens style, while projecting the content LoRA into the style null space balances better, which motivates the asymmetric design.

Limitations

• Assumes cross-layer independence of LoRA representations, which may miss cross-layer or nonlinear dependencies.
• May be less effective when content and style features are strongly entangled across layers.
• Overlap statistic (about 98%) is an empirical indicator of subspace correlation, not a direct measure of interference; the subspace analysis is illustrative rather than rigorous.

Experimental setup

Datasets: DreamBooth StyleDrop Hugging Face community LoRAs

Benchmarks: DreamBooth-based content-style pairs (32 pairs) community FLUX LoRA pairs

Metrics: CLIP content similarity CLIP style similarity DINO content similarity DINO style similarity arithmetic mean harmonic mean user preference

Baselines: Direct merging ZipLoRA DuoLoRA K-LoRA LoRA.rar B-LoRA Joint training

Builds on: LoRA ZipLoRA K-LoRA DuoLoRA LoRA.rar

Closest papers by meaning

• K-LoRA: Unlocking Training-Free Fusion of Any Subject and Style LoRAs (CVPR 2025 · similarity 0.90)
• Uni-LoRA: One Vector is All You Need (NeurIPS 2025 · similarity 0.89)
• Flat-LoRA: Low-Rank Adaptation over a Flat Loss Landscape (ICML 2025 · similarity 0.89)
• AuroRA: Breaking Low-Rank Bottleneck of LoRA with Nonlinear Mapping (NeurIPS 2025 · similarity 0.89)
• SDS-LoRA: Overcoming Anisotropic Gradient Scaling in Low-Rank Adaptation (NeurIPS 2026 · similarity 0.89)
• LoRAtorio: An intrinsic approach to LoRA Skill Composition (NeurIPS 2026 · similarity 0.88)