Research map / Continual learning / Continual fine-tuning of LLMs
MMKU-Bench: A Multimodal Update Benchmark for Diverse Visual Knowledge
Baochen Fu, Yuntao Du, Cheng Chang, Baihao Jin, Wenzhi Deng, Muhao Xu, Hongmei Yan, Weiye Song, Yi Wan
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| provide golden context via RAG | InternVL2.5-8B · MMKU-Bench Updated vs Unknown | Correct (CEM): Updated 67.02 vs Unknown 76.03 | Mixed |
| use RLHF (CPO/SimPO) instead of SFT | InternVL2.5-8B · MMKU-Bench Updated | Outdated (CEM): Full-FT 5.53 -> SimPO 3.94 | Improved |
| use RLHF instead of SFT for overall injection | InternVL2.5-8B · MMKU-Bench Updated | Correct (CEM): Full-FT 35.19 -> CPO 33.35 | No clear effect |
| use knowledge editing (WISE) for large-scale injection | InternVL2.5-8B · MMKU-Bench Updated | Correct (CEM): 35.19 -> 6.06 | Hurt |
| use GRACE editing | InternVL2.5-8B · MMKU-Bench Updated | Correct (CEM): 0.61 | Hurt |
| inject updated instead of unknown knowledge with Full-FT (ablation) | InternVL2.5-8B · MME | score: Unknown 1090.6 vs Updated 499.1 | Hurt |
| inject updated instead of unknown knowledge with Full-FT | Qwen2.5-VL-7B · MME | score: Unknown 1162.7 vs Updated 393.9 | Hurt |
| use LoRA instead of full fine-tuning (ablation) | InternVL2.5-8B · MME (Unknown) | score: Full-FT 1090.6 -> LoRA 1515.0 | Improved |
| use WISE editing to preserve capability | InternVL2.5-8B · MME | score: Vanilla 1688.2 vs WISE 1672.6 | Improved |
| fine-tune Qwen2.5-VL with LoRA on updated knowledge | Qwen2.5-VL-7B · MMBench | accuracy: 82.2 -> 9.1 | Hurt |
| match injection and evaluation modality (M->M) | SFT/RLHF models · MMKU-Bench cross-modal | Correct | Improved |
| inject multimodal but evaluate on text (M->T) | SFT/RLHF models · MMKU-Bench Updated | Correct | Hurt |
Closest papers by meaning
- MMKE-Bench: A Multimodal Editing Benchmark for Diverse Visual Knowledge (ICLR 2025 · similarity 0.92)
- MMBench-Live: A Continuously Evolving Benchmark for Multimodal Models (ICML 2026 · similarity 0.90)
- A Practitioner's Guide to Real-World Continual Multimodal Pretraining (NeurIPS 2024 · similarity 0.89)
- Benchmarking Multimodal Knowledge Conflict for Large Multimodal Models (AAAI 2026 · similarity 0.88)
- Seeking and Updating with Live Visual Knowledge (NeurIPS 2025 · similarity 0.88)
- MICON-Bench: Benchmarking and Enhancing Multi-Image Context Image Generation in Unified Multimodal Models (CVPR 2026 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
