AI Paper Atlas

Research map / Continual learning / Continual fine-tuning of LLMs

When Large Multimodal Models Confront Evolving Knowledge: Challenges and Explorations

Kailin Jiang, Yuntao Du, Yukai Ding, Yuchen Ren, Ning Jiang, Zhi Gao, Zilong Zheng, Lei Liu, Bin Li, Qing Li

ICLR 2026 · Continual learning › Continual fine-tuning of LLMs · approach: Mixture-of-experts for multimodal continual tuning · 3 citations (OpenAlex)

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
fine-tune LMM on evolving knowledge (Full-FT) instead of no injectionLLaVA-v1.5 · MMEvoke (all)CEM: 4.89 -> 18.02Improved
use multimodal RAG (UniIR) instead of fine-tuningLLaVA-v1.5 · MMEvoke (all)CEM: 18.02 -> 40.68Improved
provide sufficient context instead of retrieved contextQwen-VL-Chat · MMEvoke (all)CEM: 32.75 -> 48.96Improved
add knowledge-agnostic text augmentation to SFT (ablation)LLaVA-v1.5 SFT · MMEvoke (all)CEM: -5.66% CEM, -7.78% F1 (text)Hurt
add knowledge-agnostic image augmentation to SFT (ablation)LLaVA-v1.5 SFT · MMEvoke Entity subsetCEM: -47.36% CEM, -12.76% F1Hurt
add knowledge-aware text augmentation to SFT (ablation)LLaVA-v1.5 SFT · MMEvoke (all)CEM: +11.32% CEM, +39.82% F1Improved
add knowledge-aware visual augmentation to SFT (ablation)LLaVA-v1.5 SFT · MMEvoke Entity subsetCEM: +43.28% CEM, +13.19% F1Improved
add Replay (10% pretraining data) to LoRA injection (ablation)LLaVA-v1.5 LoRA · MIA-Bench (retention)score: 29.66 -> 62.33Improved
use MoELoRA instead of LoRA (ablation)LLaVA-v1.5 · MIA-Bench (retention)score: 29.66 -> 64.97Improved
add EWC regularization to fine-tuning (ablation)LLaVA-v1.5 · General capability testsscore: MIA-Bench 25.25 -> 29.79No clear effect
add LwF to fine-tuning (ablation)LLaVA-v1.5 · General capability testsscore: MMDU 13.03 -> 13.68Mixed
add knowledge-aware text augmentation to injection (retention effect) (ablation)LLaVA-v1.5 · General capability testsaverage degradation (ranking): ranked 4 vs Full-FT 8, LoRA 7Mixed

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy