Research map / Image generation / Text-to-image diffusion
FouRA: Fourier Low-Rank Adaptation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FouRA, a LoRA variant that applies low-rank projections in the Fourier/DCT domain with an input-dependent learned rank-gating mask, improving diversity and quality of LoRA-finetuned text-to-image diffusion models and enabling training-free adapter merging.
Problem
LoRA-finetuned text-to-image diffusion models suffer from low diversity (data copying / distribution collapse), worsened by high adapter strength, high rank, and small training sets; merging multiple LoRAs is also prone to artifacts.
Approach
Apply the low-rank down/up projections in the frequency domain (1D DFT or DCT along the embedding dimension) with a learnable, input- and timestep-dependent gating over the rank-r singular components, folding the adapter strength alpha into the low-rank subspace.
Key results
- On Paintings with Realistic Vision-v3.0 at alpha=1.0, FouRA reaches LPIPS diversity 44.2 and HPSv2 28.4 vs LoRA 38.3 and 24.6 (rank 64, 30 seeds).
- Merging BlueFire and Paintings adapters at alpha 1.0, FouRA HPSv2 is 33.1 vs LoRA 30.3, with FouRA ahead at every tested strength.
- On GLUE with RoBERTa-Base, FouRA outperforms SoRA on four of six tasks (e.g., CoLA 70.6 vs 69.9, QNLI 94.2 vs 93.9).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FouRA instead of LoRA | Stable Diffusion-v1.5 on Paintings · Paintings (α=1) | LPIPS Diversity: 38.3 ± 3.6 -> 43.9 ± 3.7 | Helped |
| Use FouRA instead of LoRA | Stable Diffusion-v1.5 on Paintings · Paintings (α=1) | HPSv2: 22.3 ± 1.7 -> 25.2 ± 1.6 | Helped |
| Use FouRA instead of LoRA | Realistic Vision-v3.0 on Paintings · Paintings (α=1) | LPIPS Diversity: 38.3 ± 3.5 -> 44.2 ± 3.7 | Helped |
| Use FouRA instead of LoRA | Realistic Vision-v3.0 on Paintings · Paintings (α=1) | HPSv2: 24.6 ± 1.8 -> 28.4 ± 1.8 | Helped |
| Use FouRA instead of LoRA | Stable Diffusion-v1.5 on Blue-Fire · Blue-Fire (α=1) | LPIPS Diversity: 47.8 ± 3.7 -> 50.3 ± 3.0 | Helped |
| Use FouRA instead of LoRA | Stable Diffusion-v1.5 on Blue-Fire · Blue-Fire (α=1) | HPSv2: 28.6 ± 2.1 -> 29.7 ± 1.9 | Helped |
| Use FouRA instead of LoRA | Realistic Vision-v3.0 on Blue-Fire · Blue-Fire (α=1) | LPIPS Diversity: 46.8 ± 4.0 -> 50.4 ± 3.0 | Helped |
| Use FouRA instead of LoRA | Realistic Vision-v3.0 on Blue-Fire · Blue-Fire (α=1) | HPSv2: 32.7 ± 1.6 -> 33.6 ± 1.5 | Helped |
| Use FouRA instead of LoRA | RoBERTa-Base · CoLA | MCC: 67.3 ± 0.8 -> 70.6 ± 0.7 | Helped |
| Use FouRA instead of LoRA | RoBERTa-Base · SST2 | accuracy: 94.9 ± 0.3 -> 95.5 ± 0.4 | Helped |
| Use FouRA instead of LoRA | RoBERTa-Base · STSB | correlation: 89.9 ± 0.3 -> 91.6 ± 0.1 | Helped |
| Use FouRA instead of LoRA | RoBERTa-Base · MRPC | score: 90.3 ± 0.6 -> 90.4± 0.5 | No clear effect |
Surprising findings
- FouRA's HPSv2 remains above the best LoRA score across all ranks, suggesting the frequency-domain gating reduces the rank sensitivity of LoRA.
Limitations
- Experiments on small style datasets (30-90 images) and a limited set of styles and concepts.
- Evaluation relies on learned metrics (HPSv2.1, LPIPS diversity) rather than human studies.
Experimental setup
Datasets: BlueFire Paintings Origami 3D GLUE concept slider attribute prompts
Benchmarks: GLUE BlueFire validation Paintings validation
Metrics: LPIPS diversity HPSv2.1 LPIPS similarity accuracy Matthews correlation Pearson/Spearman correlation
Baselines: LoRA SoRA AdaLoRA
Builds on: LoRA SoRA concept sliders soft gating
Closest papers by meaning
- GraLoRA: Granular Low-Rank Adaptation for Parameter-Efficient Fine-Tuning (NeurIPS 2025 · similarity 0.88)
- Flat-LoRA: Low-Rank Adaptation over a Flat Loss Landscape (ICML 2025 · similarity 0.88)
- LoRAtorio: An intrinsic approach to LoRA Skill Composition (NeurIPS 2026 · similarity 0.88)
- RandLoRA: Full rank parameter-efficient fine-tuning of large models (ICLR 2025 · similarity 0.88)
- Decoupling Angles and Strength in Low-rank Adaptation (ICLR 2025 · similarity 0.88)
- Parameter-Efficient Fine-Tuning with Discrete Fourier Transform (ICML 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.