Research map / Vision-language and multimodal / Multimodal LLMs

Cross-Modal Unlearning via Influential Neuron Path Editing in Multimodal Large Language Models

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Benchmarks and analysis of MLLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes MIP-Editor, which locates modality-specific influential neuron paths in MLLM FFN layers via inter-layer gradient/Fisher integration, prunes them, and repairs them with representation-misdirection unlearning (RMisU) to forget multimodal and textual knowledge while preserving general ability.

Problem

Machine unlearning in multimodal LLMs suffers from inconsistent forgetting across modalities (especially text) and from utility loss when pruning neurons that also support general reasoning; point-wise neuron attribution fails to capture cross-layer information flow.

Approach

Replace point-wise neuron scoring with path-level attribution: inter-layer gradient-integrated (IGI) scores for text and inter-layer Fisher-integrated (IFI) scores for vision identify a greedy layer-wise neuron path; only those path neurons are zeroed and then edited with RMisU, which maps forget-set hidden states to random directions while anchoring retain-set states to the frozen model.

Key results

• On MLLMU-Bench with Qwen2.5-VL-3B at 5% forget ratio, MIP-Editor lowers FVQA accuracy from 39.20% to 4.80% (reported as 87.75% forgetting) and raises RVQA from 37.72% to 58.19%.
• On textual FQA, MIP-Editor reduces accuracy from 49.60% to 9.60% (reported 80.65% forgetting) while retaining 77.9% of general performance.
• Ablations show IGI-only or IFI-only path localization and removing RMisU editing each weaken forgetting and retention; path-based selection retains performance with fewer neurons than point-wise selection.

Surprising findings

• Pruning-only and full-model RMisU variants degrade retain-set performance sharply (e.g., RVQA 2.11% for path pruning without editing), indicating that pruning alone harms general knowledge.

Limitations

• CLEAR results are unstable and treated as supplementary; the authors note model collapse under minor hyperparameter changes.
• Baseline and MIP-Editor results show large variance in some settings (e.g., LLaVA CLEAR RVQA and FVGEN results), and ROUGE-L is weak for generation tasks.
• Evaluated only on two MLLMs and two benchmarks with fictitious/synthetic profiles.

Experimental setup

Datasets: MLLMU-Bench CLEAR

Benchmarks: MLLMU-Bench CLEAR

Metrics: accuracy ROUGE-L

Baselines: GA_Diff KL_Min NPO MANU

Builds on: MANU representation misdirection unlearning NPO integrated gradients

Closest papers by meaning

• Towards Neuron Attributions in Multi-Modal Large Language Models (NeurIPS 2024 · similarity 0.91)
• Towards Reasoning-Preserving Unlearning in Multimodal Large Language Models (CVPR 2026 · similarity 0.89)
• Edit Less, Achieve More: Dynamic Sparse Neuron Masking for Lifelong Knowledge Editing in LLMs (NeurIPS 2025 · similarity 0.88)
• Learning What to Forget: Improving LLM Unlearning via Learned Token-Level Importance (NeurIPS 2026 · similarity 0.88)
• LoRASculpt: Sculpting LoRA for Harmonizing General and Specialized Knowledge in Multimodal Large Language Models (CVPR 2025 · similarity 0.88)
• ASRU: Activation Steering Meets Reinforcement Unlearning for Multimodal Large Language Models (ICML 2026 · similarity 0.88)