Research map / Vision-language and multimodal / Multimodal LLMs

AUVIC: Adversarial Unlearning of Visual Concepts for Multi-modal Large Language Models

AAAI 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Benchmarks and analysis of MLLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes AUVIC, an adversarial visual-concept unlearning method for MLLMs (LLaVA-1.5 7B) using a CLIP-feature-driven perturbation generator, LoRA on the vision tower, and Gumbel-Softmax dynamic anchor preservation, and introduces the VCUBench benchmark of public figures in single and group photos.

Problem

Removing a specific visual concept (identity of a person) from a multimodal LLM without degrading recognition of similar entities or general performance, i.e., avoiding collateral forgetting in group photos.

Approach

A min-max adversarial unlearning loop in which a lightweight generator, fed frozen CLIP features, perturbs images to strongly activate the target concept while the vision tower (via LoRA) learns to suppress it; a Gumbel-Softmax-sampled set of similar public figures is protected with a preservation loss, and a KL consistency term regularizes outputs.

Key results

• On the Trump target, AUVIC reaches TFA 93.64% and NTRA 83.17% with GRF-F1 88.10%, versus 75.25% GRF-F1 for SIU and 44.46% for GA.
• Averaged over six target concepts, AUVIC achieves TFA 96.99, NTRA 75.34, GRF-F1 84.94 and perplexity 8.34, best on all reported metrics.
• Ablation: removing both adversarial perturbation and Gumbel sampling drops GRF-F1 to 38.55 (TFA 27.43), while removing only the adversarial perturbation gives GRF-F1 70.98.

Surprising findings

• Removing both components (w/o Both) yields a high NTRA of 75.43 but very low TFA of 27.43, indicating the forgetting drop is driven mainly by the adversarial components.
• The paper's own SIU baseline reaches 100% Efficacy and higher TFA than AUVIC on the Trump table (92.35% vs 93.64% is lower; but Efficacy 100.0% exceeds AUVIC's 97.92%).

Experimental setup

Datasets: VCUBench ScienceVQA

Benchmarks: VCUBench ScienceVQA

Metrics: TFA NTRA GRF-F1 Efficacy Generality Perplexity Recall BLEU

Baselines: Gradient Ascent (GA) Preference Optimization (PO) GA+KL SIU

Builds on: LLaVA-1.5 CLIP LoRA TOFU

Closest papers by meaning

• Single Image Unlearning: Efficient Machine Unlearning in Multimodal Large Language Models (NeurIPS 2024 · similarity 0.92)
• Defensive Unlearning with Adversarial Training for Robust Concept Erasure in Diffusion Models (NeurIPS 2024 · similarity 0.89)
• Towards Robust Knowledge Unlearning: An Adversarial Framework for Assessing and Improving Unlearning Robustness in Large Language Models (AAAI 2025 · similarity 0.88)
• MLUBench: A Benchmark for Lifelong Unlearning Evaluation in MLLMs (ICML 2026 · similarity 0.88)
• On Effects of Steering Latent Representation for Large Language Model Unlearning (AAAI 2025 · similarity 0.88)
• Wisdom is Knowing What not to Say: Hallucination-Free LLMs Unlearning via Attention Shifting (NeurIPS 2025 · similarity 0.88)