Research map / Image generation / Editing and style transfer
Time-Varying LoRA: Towards Effective Cross-Domain Fine-Tuning of Diffusion Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes Terra, a time-varying low-rank adapter for SDXL that interpolates between domains via a continuous time variable, and uses it to generate domain-flow samples that augment UDA and DG methods.
Problem
Standard LoRA needs multiple adapters to represent multiple domains and cannot smoothly generate intermediate domains, which limits cross-domain generation and domain shift handling in UDA and DG.
Approach
Insert a time-dependent middle matrix K(t)=F(W_mid,t) into the low-rank structure so one adapter traces a continuous parameter manifold, with t=0 and t=1 representing two domains; generated intermediate domains then serve as augmentation for UDA and DG.
Key results
- Terra achieves FID 62.25 on image-pair morphing versus 78.26 for DiffMorpher and 89.37 for LoRA Interpolation.
- ELS+Terra reaches 76.26% average on Office-Home UDA versus 71.84% for ELS alone.
- ERM+Terra reaches 86.50% average on PACS DG versus 84.41% for ERM.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add Terra to ELS | ELS · Office-Home UDA | accuracy: 71.84 -> 76.26 | Helped |
| Add Terra to ELS | ELS · VisDA UDA | accuracy: 83.40 -> 86.86 | Helped |
| Add Terra to ELS | ELS · Office31 UDA | accuracy: 90.21 -> 91.28 | Helped |
| Add Terra to SWAD | SWAD · PACS DG | accuracy: 88.27 -> 89.34 | Helped |
| Add Terra to SWAD | SWAD · Office-Home DG | accuracy: 70.88 -> 72.23 | Helped |
| Use adapted source and generated target domains together rather than the original source alone (ablation) | ELS+Terra · Office-Home UDA | accuracy: 71.84 -> 76.26 | Helped |
| Use adapted source domain only instead of the expanded domain (ablation) | ELS+Terra · Office-Home UDA | accuracy: 76.26 -> 72.92 | Hurt |
| Use generated target domain only instead of the expanded domain (ablation) | ELS+Terra · Office-Home UDA | accuracy: 76.26 -> 72.18 | Hurt |
| Set the time-variable dimension to 2 rather than 1 (ablation) | ERM+Terra · PACS DG | accuracy: 85.37 -> 86.50 | Helped |
| Set the time-variable dimension to 3 rather than 2 (ablation) | ERM+Terra · PACS DG | accuracy: 86.50 -> 86.26 | Hurt |
| Use the Cosine-Sine form instead of the Linear form (ablation) | ERM+Terra · PACS DG | accuracy: 85.66 -> 86.50 | Helped |
Surprising findings
- Terra's morphing results are not uniformly best; on PPL for image pairs, DiffMorpher (0.77) beats Terra (0.95), and Terra's FID on styles is better than LoRA Interpolation but no DiffMorpher comparison is given there.
Limitations
- Experiments are limited to image domains and relatively small benchmark datasets; the authors note extension to other modalities and larger datasets is future work.
- Choice of the time-dependent function F (linear, exponential, cosine-sine) and time dimension is task-dependent and chosen empirically.
Experimental setup
Datasets: Office31 Office-Home VisDA PACS VLCS DiffMorpher image repository
Benchmarks: Office31 Office-Home VisDA PACS VLCS
Metrics: FID PPL transfer accuracy classification accuracy
Baselines: DGP DDIM inversion LoRA Interpolation DiffMorpher ERM MCC ELS MSGD SAGM SWAD DomainDiff CDGA
Builds on: LoRA DiffMorpher DomainDiff MCC ELS SWAD
Closest papers by meaning
- LoRA-X: Bridging Foundation Models with Training-Free Cross-Model Adaptation (ICLR 2025 · similarity 0.89)
- Smooth Diffusion: Crafting Smooth Latent Spaces in Diffusion Models (CVPR 2024 · similarity 0.88)
- TimeStep Master: Asymmetrical Mixture of Timestep LoRA Experts for Versatile and Efficient Diffusion Models in Vision (ICML 2025 · similarity 0.88)
- LoRAtorio: An intrinsic approach to LoRA Skill Composition (NeurIPS 2026 · similarity 0.88)
- Image is All You Need to Empower Large-scale Diffusion Models for In-Domain Generation (CVPR 2025 · similarity 0.87)
- Zero-Shot Adaptation of Parameter-Efficient Fine-Tuning in Diffusion Models (ICML 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.