Research map / Continual learning / Continual fine-tuning of LLMs
Mixing Expertise with Confidence: A Mixture of Experts Framework for Robust Multi-Modal Continual Learning
Md Abdullah Al Forhad, Yuansheng Zhu, Abhinab Acharya, Xumin Liu, Qi Yu, Weishi Shi
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add adaptive threshold temperature scaling to independently trained experts (ablation) | Expert mixture · CIFAR-100 10 steps | task ID accuracy | Improved |
| add Mahalanobis-distance weighting only (no temperature) (ablation) | Expert mixture · CIFAR-100 10 steps | task ID accuracy | Improved |
| combine temperature scaling and MD weighting (ablation) | Expert mixture · CIFAR-100 10 steps | task ID accuracy | Improved |
| apply temperature scaling to task-ID detection (ablation) | Expert mixture · CIFAR-100 10 steps | per-task detection | Mixed |
| scale temperature on shortcut-activated samples (ablation) | Expert · CIFAR-100 10 steps | mean predictive variance: 0.007-0.010 range -> 0.0005-0.0015 range | Improved |
| remove shared parameters and use confidence-based expert communication | Ours vs MoE-Adapters · CIFAR-100 10 steps | last accuracy: 77.52 -> 80.6 | Improved |
| use multimodal expert mixture instead of RAIL on subpopulation shift | Ours vs RAIL · BREEDS Entity-13 (5 steps) | All accuracy: 81.06 -> 87.08 | Improved |
| use same mixture on 50-step CIFAR-100 | Ours vs RAIL · CIFAR-100 50 step | average accuracy: 85.11 -> 85.68 | Improved |
Closest papers by meaning
- Theory on Mixture-of-Experts in Continual Learning (ICLR 2025 · similarity 0.93)
- FLAME: Adaptive Mixture-of-Experts for Continual Multimodal Multi-Task Learning (NeurIPS 2026 · similarity 0.91)
- Orchestrate Latent Expertise: Advancing Online Continual Learning with Multi-Level Supervision and Reverse Self-Distillation (CVPR 2024 · similarity 0.89)
- Training Consistent Mixture-of-Experts-Based Prompt Generator for Continual Learning (AAAI 2025 · similarity 0.88)
- Divide and not forget: Ensemble of selectively trained experts in Continual Learning (ICLR 2024 · similarity 0.87)
- Learn more, but bother less: parameter efficient continual learning (NeurIPS 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
