AI Paper Atlas

Research map / Large language models / Language, safety and interpretability

Multiple Choice Learning of Low-Rank Adapters for Language Modeling

Victor Letzelter, Hugo Malard, Mathieu Fontaine, Gaël Richard, Slim Essid, Andrei Bursuc, Patrick Perez

ICML 2026 · Large language models › Language, safety and interpretability · approach: LLM pruning and scaling data

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add multiple LoRA hypotheses trained with WTA (LoRA-MCL, K=7) instead of single LoRA-MLE (r=8)Qwen2-Audio · AudioCaps test losstest loss (NLL): 2.203 -> 1.932Improved
add multiple LoRA hypotheses trained with WTA (LoRA-MCL, K=7) instead of single LoRA-MLE (r=8)Qwen2-Audio · Clotho test losstest loss (NLL): 2.812 -> 2.612Improved
increase number of hypotheses K from 3 to 7 (ablation)LoRA-MCL · AudioCaps test losstest loss (NLL): 2.063 -> 1.932Improved
increase number of hypotheses K from 3 to 7 (ablation)LoRA-MCL · Clotho test losstest loss (NLL): 2.663 -> 2.612Improved
increase LoRA-MLE rank from 8 to 8x7 for parameter parity (ablation)LoRA-MLE · Clotho test losstest loss (NLL): 2.812 -> 2.935Hurt
increase LoRA-MLE rank from 8 to 8x7 for parameter parity (ablation)LoRA-MLE · AudioCaps test losstest loss (NLL): 2.203 -> 2.182No clear effect
replace LoRA-MLE with DBS (tuned lambda) by LoRA-MCL at equal forward passesLLaVA 1.6 · TextCaps valSPIDEr: 0.926 -> 0.955Improved
use LoRA-MCL greedy instead of LoRA-MLE with DBSLLaVA 1.6 · TextCaps valmBLEU-4 (lower is more diverse): 0.416 -> 0.520Hurt
use LoRA-MCL instead of LoRA-MLE with DBS on bilingual synthetic data (French)LLaVA 1.6 · TextCaps bilingual (French)SPIDEr: 0.411 -> 0.464Improved
use LoRA-MCL instead of LoRA-MLE with DBS on bilingual synthetic data (diversity, French)LLaVA 1.6 · TextCaps bilingual (French)mBLEU-4 (lower is more diverse): 0.138 -> 0.027Improved
use LoRA-MCL instead of LoRA-MLE with DBS on bilingual synthetic data (diversity, English)LLaVA 1.6 · TextCaps bilingual (English)mBLEU-4 (lower is more diverse): 0.126 -> 0.029Improved
increase beam size within each group of diverse beam search (ablation)LoRA-MLE and LoRA-MoE · TextCaps val / Clothoquality and diversityNo clear effect

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy