AI Paper Atlas

Research map / Continual learning / Continual fine-tuning of LLMs

Spectral Mixture-of-Experts for Continual Learning

Chen Yin, Xingbo Dong, Xuelin Shen, Zhe Jin

CVPR 2026 · Continual learning › Continual fine-tuning of LLMs · approach: Low-rank and model-merging adaptation

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add router consistency projection Hw (ablation)Spectral MoE · MTIL full-shot (11 tasks)Average accuracy (%): 75.1 -> 78.1Improved
add dynamic expert projection He(Dη) (ablation)Spectral MoE · MTIL full-shot (11 tasks)Last accuracy (%): 79.8 -> 86.3Improved
add both DCP components on top of Spectral Experts (ablation)Spectral MoE · MTIL full-shot (11 tasks)Average accuracy (%): 75.0 -> 78.1Improved
replace LoRA experts with Spectral Experts (ablation)Spectral MoE · Forgetting AUC, full-shotForgetting AUC (%): 1.81% -> 0.91%Improved
replace LoRA experts with Spectral Experts (ablation)Spectral MoE · Forgetting AUC, few-shotForgetting AUC (%): 1.68% -> 0.61%Improved
use full stability Dη = I instead of dynamic Dη (ablation)Spectral MoE · MTIL full-shot (11 tasks)Transfer accuracy (%): 70.1 -> 68.7Hurt
use full plasticity Dη = 0 instead of dynamic Dη (ablation)Spectral MoE · MTIL full-shot (11 tasks)Last accuracy (%): 86.3 -> 80.3Hurt
set stability floor eta_min to 0 instead of 0.95 (ablation)Spectral MoE · MTIL full-shot (11 tasks)accuracyHurt
use focusing strength gamma 1 or >=3 instead of 2 (ablation)Spectral MoE · MTIL full-shot (11 tasks)accuracyHurt
increase number of experts up to N=32 (ablation)Spectral MoE · MTIL full-shot (11 tasks)accuracyImproved
use Spectral Experts instead of LoRA experts in MoE adapterMoE adapters · Training costTrainable parameters: 59.8M -> 23.5 MImproved
use Spectral MoE instead of MoE-Adapters++CLIP continual learning · MTIL full-shot (11 tasks)Average accuracy (%): 77.5 -> 78.1Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy