AI Paper Atlas

Research map / Architectures and efficiency / Pruning and efficient tuning

LoRAGen: Structure-Aware Weight Space Learning for LoRA Generation

Hao Huang, Jingtao Ding, Mengqi Liao, Xin Wang, Jinyang Ban, Yuan Yuan, Huaiyu Wan, Yong Li

ICLR 2026 · Architectures and efficiency › Pruning and efficient tuning · approach: LoRA variants and theory

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add module-aware MoE decoder (ablation)LoRAGen · FLAN subset (in-distribution)average accuracy: 58.4 -> 96.0Improved
add direction and spectral adapter-level losses to the MoE decoder (ablation)LoRAGen · FLAN subset (in-distribution)average accuracy: 95.2 -> 96.0Improved
add direction loss alongside the spectral loss (ablation)LoRAGen · FLAN subset (in-distribution)average accuracy: 36.9 -> 96.0Improved
add adapter-level losses to the decoder for unseen tasks (ablation)LoRAGen · FLAN subset (zero-shot)average accuracy: 36.6 -> 40.2Improved
use LoRAGen instead of T2L on unseen tasksLoRAGen · FLAN subset (zero-shot)average accuracy: 35.2 -> 40.2Improved
use LoRAGen instead of T2L on in-distribution tasksLoRAGen · FLAN subset (in-distribution)average accuracy: 88.7 -> 96.0Improved
use LoRAGen vs task-specific LoRAs on Gemma-2-2B HellaSwagLoRAGen · HellaSwag (Gemma-2-2B-Instruct)accuracy: 75.4 -> 64.1Hurt
use LoRAGen vs task-specific LoRAs on Gemma-2-2B averageLoRAGen · 8 benchmark tasks (Gemma-2-2B-Instruct)average accuracy: 74.5 -> 72.7Hurt
add structural embedding to the MoE decoder routing (ablation)LoRAGen · FLAN subset (in-distribution)accuracyImproved
use unshared expert pools instead of shared pools (ablation)LoRAGen MoE decoder · FLAN subset (in-distribution)accuracyImproved
increase active experts from top-1 to top-2 (ablation)LoRAGen MoE decoder · FLAN subset (in-distribution)accuracyImproved
vary spectral-energy threshold rho from 0.80 to 1.00 (ablation)LoRAGen spectral loss · FLAN subset (in-distribution)accuracyNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy