AI Paper Atlas

Research map / Continual learning / Continual fine-tuning of LLMs

SAME: Stabilized Mixture-of-Experts for Multimodal Continual Instruction Tuning

Zhen-Hao Xie, Jun-Tao Tang, Yu-Cheng Shi, Han-Jia Ye, De-Chuan Zhan, Da-Wei Zhou

ICML 2026 · Continual learning › Continual fine-tuning of LLMs · approach: Mixture-of-experts for multimodal continual tuning · 0 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add spectral-aware routing to MoELoRA (ablation)MoELoRA · CoIN (8 tasks)average accuracy: 50.58 -> 61.32Improved
add curvature-aware Riemannian scaling on top of spectral-aware routing (ablation)SAME · CoIN (8 tasks)average accuracy: 61.32 -> 65.89Improved
add adaptive expert activation (freeze selected experts) on top of routing and scaling (ablation)SAME · CoIN (8 tasks)average accuracy: 65.89 -> 66.82Improved
add adaptive expert activation (ablation)SAME · CoIN (8 tasks)training time per task: -32.1 min per task, -2.3K MiB/GPU on averageImproved
add adaptive expert activation (ablation)SAME · CoIN OCR-VQA and ScienceQA tasksper-task accuracy: ScienceQA 80.29 -> 78.35; OCR-VQA 63.19 -> 63.59Mixed
add curvature-aware scaling on top of routing (ablation)SAME · CoIN VQAv2 and OCR-VQA tasksper-task accuracy: VQAv2 64.68 -> 61.21; ScienceQA 71.44 -> 80.29Mixed
add spectral-aware routing (ablation)MoELoRA · CoIN Task 1 router overlap after Task 8routing overlap (%): 79.40 -> 81.59Improved
add curvature-aware scaling with router re-trained (ablation)SAME · CoIN Task 1 recovered accuracyre-routing accuracyImproved
use SAME instead of MoELoRAMoELoRA · TriGap (10 tasks)average accuracy: 44.45 -> 46.53Improved
use SAME instead of MoELoRAMoELoRA · ScienceQA case-mismatch error after Task 8case mismatch error rate (%): 22.5 -> 7.2Improved

Earlier methods this paper compares with

Named baselines from earlier accepted papers in this paper's results tables: better than 2 of 2 on most shared results.

Matched by method name in results tables read by AI. Authors' own tables tend to favour the newer method, and a comparison can be missed when a method is renamed.

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy