AI Paper Atlas

Research map / Architectures and efficiency / Mixture of experts

LoRA-Mixer: Coordinate Modular LoRA Experts Through Serial Attention Routing

Wenbing Li, Zikai Song, Hang Zhou, Junqing Yu, Yunyao Zhang, Wei Yang

ICLR 2026 · Architectures and efficiency › Mixture of experts · approach: Mixture-of-experts LoRA · 23 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add RSL (entropy-regularized balance loss) instead of auxiliary loss, 1K routing data (ablation)LoRA-Mixer · 7-task average (1K routing data)average accuracy: 75.47 -> 76.80Improved
add RSL instead of auxiliary loss, 2K routing data (ablation)LoRA-Mixer · 7-task average (2K routing data)average accuracy: 77.29 -> 79.26Improved
add RSL instead of auxiliary loss, 4K routing data (ablation)LoRA-Mixer · 7-task average (4K routing data)average accuracy: 79.14 -> 78.77Hurt
add RSL instead of auxiliary loss, 10K routing data (ablation)LoRA-Mixer · 7-task average (10K routing data)average accuracy: 79.51 -> 79.94No clear effect
use RSL instead of AESL routing lossLoRA-Mixer · HumanEvalPass@1: 50.46 -> 57.32Improved
use RSL instead of GMoE routing lossLoRA-Mixer · ARC-Caccuracy: 76.42 -> 83.24Improved
use alpha=0.1, lambda=0.05 instead of alpha=0.2, lambda=0.1 (ablation)LoRA-Mixer · alpha/lambda sweepperformance (%): 75.8% -> 78.5%Improved
use alpha=0.1, lambda=0.05 instead of alpha=0.05, lambda=0.01 (ablation)LoRA-Mixer · alpha/lambda sweepperformance (%): 76.2% -> 78.5%Improved
compose LoRAs with LoRA-Mixer over single-task LoRALoRA-Mixer · BoolQaccuracy: 74.46 -> 79.37Improved
compose LoRAs with LoRA-Mixer over LoRA-LEGO on RTELoRA-Mixer · RTE (LLaMA2-7B)accuracy: 71.85 -> 61.47Hurt
compose LoRAs with LoRA-Mixer over LoRA-LEGO on CoLALoRA-Mixer · CoLA (LLaMA2-7B)accuracy: 55.48 -> 64.60Improved
compose LoRAs with LoRA-Mixer over single-task LoRA on GSM8KLoRA-Mixer · GSM8K (Mistral-7B)accuracy: 46.67 -> 46.48Hurt

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy