AI Paper Atlas

Research map / Vision-language and multimodal / Multimodal LLMs

Knowledge Externalization: Reversible Unlearning and Modular Retrieval in Multimodal Large Language Models

Jiaqi Li, Zihan You, Ruoyan Shen, Shenyu Zhang, Songlin Zhai, Yongrui Chen, Chuanyi Zhang, Jiahui Geng, Fakhri Karray, Sheng Bi, Guilin Qi

ICLR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Unified any-modality language models

Read the paperCheck your idea against 137,217 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes knowledge externalization, which moves sensitive knowledge out of MLLM parameters into a modular, reversible store so it can be unlearned and retrieved on demand.

Problem

Machine unlearning in multimodal LLMs irreversibly alters parameters to erase sensitive personal or proprietary knowledge, which conflicts with privacy regulations requiring auditable and reversible handling of data.

Approach

Instead of permanently erasing knowledge from weights, externalize it into a separate modular component that can be removed from the model (unlearning) and later reattached via retrieval (reversibility), so the base parameters stay intact.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
add Soft Orthogonal Weighting to DSM for triple-concept externalization (ablation)DSM · MEXBench Trump&Chihuahua&MuskGEN: 34.0 -> 97.0Improved
add Soft Orthogonal Weighting to DSM for triple-concept externalization (ablation)DSM · MEXBench Trump&Hello Kitty&Harry PotterGEN: 64.7 -> 92.7Improved
add Soft Orthogonal Weighting to DSM for dual-concept externalization (ablation)DSM · MEXBench Trump&Chihuahua REC2 (InternVL3 2B)REC2: 24 -> 90Improved
train memory token and model simultaneously (DSM) rather than sequential forget-then-recover (SFR)externalization training · MEXBench single Trump (LLaVA 7B)REC: 6 -> 100Improved
update both streams in the same step (DSM) rather than alternating tuning (AT)externalization training · MEXBench single Elon Musk (LLaVA 7B)GEN: 51 -> 91Improved
add SOW with lambda=0 (no regularization) versus lambda=0.5 (ablation)DSM w/ SOW · Trump&ChihuahuaAVG_REC: 36.5 -> 99.5Improved
increase memory token length from 32 to 100-128 (single concept) (ablation)DSM · Trump single conceptGEN: 4 -> 100Improved
increase memory token length to 256 (single concept) (ablation)DSM · Trump single conceptGEN: 100 -> 55Hurt
increase number of externalized concepts from 2 to 8 (fixed training steps) (ablation)DSM w/ SOW · MEXBench multi-conceptGEN and AVG_REC: GEN 97.0 -> 85.3 (2 vs 8 concepts)Hurt
reverse the order of concatenated memory tokens (ablation)token compositing · Trump&ChihuahuaREC1: 100 -> 5Hurt
externalize with SOW versus gradient-ascent-only SFR on general benchmarkInternVL-7B · MMBench (Trump&Chihuahua)accuracy: 66.2 -> 76.2Improved
edit externalized memory token to a new target (Chihuahua to Samoyed vs Trump to Biden)memory token editing · Knowledge editingedit success: 100 (Biden) vs 75 (Samoyed)Mixed

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy