Research map / Architectures and efficiency / Pruning and efficient tuning

LoRA-X: Bridging Foundation Models with Training-Free Cross-Model Adaptation

ICLR 2025 · Architectures and efficiency › Pruning and efficient tuning · approach: LoRA variants and theory · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes LoRA-X, a LoRA variant whose updates live in the source base model's SVD subspace so adapters can be transferred training-free to other base models (SD-v1.5/SDXL families) via subspace projection.

Problem

LoRA adapters are tied to their base model; when the base model is replaced, adapters must be retrained, which is infeasible without original or synthetic training data.

Approach

Constrain the adapter's weight update to the truncated left/right singular subspaces of the frozen base weight and train only the singular-value update; to transfer, project the source adapter onto the target model's singular subspaces (closed form, no training), applying it only to layers with high subspace similarity, selected via an optimal-transport transferability cost.

Key results

• On BlueFire, transferred LoRA-X from SDXL to SSD-1B reaches HPSv2 0.300 vs 0.316 trained (-5.3%) and DINOv2 0.969, while transfer from SD-v1.5 to SD Eff-v1.0 yields HPSv2 0.307 vs 0.296 trained.
• Without subspace projection (copying singular value updates), transferred LoRA-X HPSv2 drops to 0.1235 versus 0.3073 with projection.
• LoRA transferred without the subspace constraint loses up to 27.8% HPSv2 at rank 32, whereas LoRA-X (320 singular values, 0.16 MB) transfers with +3.7% HPSv2.

Measured effects reported in the paper

Surprising findings

• Transferred LoRA-X from a smaller source (SD Eff-v1.0 or SSD-1B) to a larger target performs close to trained adapters.
• Transferred LoRA-X outperforms trained-from-scratch on some Origami HPSv2 cases (e.g., RealVisXL +9.7%).

Limitations

• Transfer across different architecture families (e.g., SD vs SDXL) is poor according to the transferability cost metric.
• Some layers have low subspace similarity and must be filtered out, requiring manual block selection (e.g., SSD-1B blocks).
• Diversity drops for some transfers (e.g., Paintings on RealVisXL -9.6% LPIPS) and experiments are limited to style adapters on diffusion models plus one small LLM task.

Experimental setup

Datasets: BlueFire Paintings Origami awesome chatgpt prompts Pokemon

Benchmarks: BlueFire validation prompts Paintings validation prompts Origami

Metrics: HPSv2 LPIPS diversity DINOv2 BLEU ROUGE

Baselines: LoRA X-Adapter DoRA FouRA Copying singular value modifications (no subspace projection)

Builds on: LoRA SVDiff DoRA FouRA X-Adapter

Closest papers by meaning

• LoRA-GA: Low-Rank Adaptation with Gradient Approximation (NeurIPS 2024 · similarity 0.92)
• LoRAtorio: An intrinsic approach to LoRA Skill Composition (NeurIPS 2026 · similarity 0.92)
• Flat-LoRA: Low-Rank Adaptation over a Flat Loss Landscape (ICML 2025 · similarity 0.91)
• LoRA Recycle: Unlocking Tuning-Free Few-Shot Adaptability in Visual Foundation Models by Recycling Pre-Tuned LoRAs (CVPR 2025 · similarity 0.91)
• LoRA Training Provably Converges to a Low-Rank Global Minimum Or It Fails Loudly (But it Probably Won't Fail) (ICML 2025 · similarity 0.90)
• LoRA-Pro: Are Low-Rank Adapters Properly Optimized? (ICLR 2025 · similarity 0.90)