Research map / Large language models / Instruction and fine-tuning
MoLoRA: Boosting LLM-based End-to-end Speech Translation with Mixture of Low-rank Experts
Hao Zhang, Yaqi Chen, Nianwen Si, XuKui Yang, Wenlin Zhang, Dan Qu
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| replace single LoRA with mixture of low-rank experts (ablation) | LoRA · MuST-C 8 pairs avg | BLEU: 27.9 -> 32.2 | Improved |
| replace full fine-tuning with MoLoRA (ablation) | Full fine-tuning · MuST-C 8 pairs avg | BLEU: 31.9 -> 32.2 | Improved |
| add sentence-level representation (MGRF) (ablation) | MoLoRA · MuST-C En-De | BLEU: 30.4 -> 30.8 | Improved |
| use cross-attention multi-granularity fusion instead of simple summation (ablation) | MoLoRA · MuST-C En-De | BLEU: 30.5 -> 30.8 | Improved |
| add shared expert (ablation) | MoLoRA · MuST-C En-De | BLEU: 30.6 -> 30.8 | Improved |
| add load balancing loss (ablation) | MoLoRA · MuST-C En-De | BLEU: 30.6 -> 30.8 | Improved |
| increase LoRA rank r (ablation) | LoRA · MuST-C En-De | BLEU | No clear effect |
| increase number of routed experts N' up to 4 (ablation) | MoLoRA · MuST-C En-De | BLEU | Improved |
| increase routed experts beyond 4 (ablation) | MoLoRA · MuST-C En-De | BLEU | Hurt |
| use end-to-end MoLoRA instead of cascaded ST (ablation) | MoLoRA · MuST-C En-De | BLEU: 29.5 -> 30.8 | Improved |
Closest papers by meaning
- BSLoRA: Enhancing the Parameter Efficiency of LoRA with Intra-Layer and Inter-Layer Sharing (ICML 2025 · similarity 0.90)
- QA-LoRA: Quantization-Aware Low-Rank Adaptation of Large Language Models (ICLR 2024 · similarity 0.90)
- Uni-LoRA: One Vector is All You Need (NeurIPS 2025 · similarity 0.89)
- AltLoRA: Towards Better Gradient Approximation in Low-Rank Adaptation with Alternating Projections (NeurIPS 2025 · similarity 0.89)
- Batched Low-Rank Adaptation of Foundation Models (ICLR 2024 · similarity 0.89)
- Beyond Higher Rank: Token-wise Input-Output Projections for Efficient Low-Rank Adaptation (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
