Research map / Vision-language and multimodal / Multimodal LLMs

SineProject: Machine Unlearning for Stable Vision-Language Alignment

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: VLM capability analysis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SineProject, which adds sinusoidally reparameterized trainable offsets to a frozen MLLM projector to keep its Jacobian well-conditioned during unlearning, reducing over-refusal on SafeEraser and improving MLLMU-Bench forget-retain trade-offs on LLaVA-1.5.

Problem

Unlearning in multimodal LLMs (forgetting unsafe or private knowledge) destabilizes vision-language alignment, causing over-forgetting such as refusing both harmful and benign queries, which the paper attributes to ill-conditioned projector Jacobians.

Approach

Augment the frozen projector MLP with trainable additive offsets passed through sin(), i.e., W + sin(ΔW), which bounds the parameter perturbation and, per the authors' theorem, keeps the parameter Jacobian better conditioned than a standard projector during unlearning.

Key results

• On SafeEraser LLaVA-7B (PO+PD), SineProject reports SARR 25.8 vs 30.3 for SafeEraser PO+PD, with 100 refusal efficacy and ROUGE 65.8 vs 65.4.
• On SafeEraser LLaVA-13B, SineProject reduces SARR to 25.1 from 27.3 for the best baseline while keeping ~99.8 efficacy.
• Reported projector condition numbers stay below 10^3 during unlearning, versus above 10^6 for SafeEraser's second layer (self-reported).

Surprising findings

• Unregularized gradient-based baselines (GD, KL, PO) reach near 100% SARR, i.e., refuse nearly all benign queries.
• The bias term b shows no notable improvement, so the method is effectively a dense adapter on projector weights.

Limitations

• Specifically optimized for MLP projectors; cross-attention architectures like Flamingo are not addressed.
• Geometric conditioning does not resolve semantic entanglement; capacity-forgetting trade-off remains beyond ~25% deletion.
• Adversarial fine-tuning after unlearning may recover forgotten information; no formal guarantees.

Experimental setup

Datasets: SafeEraser MLLMU-Bench

Benchmarks: SafeEraser MLLMU-Bench

Metrics: ASR Refusal Rate ROUGE GPT-Eval Specificity SARR Cls accuracy Fct Cloze accuracy MIR

Baselines: GA GD GD+PD KL KL+PD PO SafeEraser (PO+PD) NPO MMUnlearner Gradient Difference KL Minimization Prompting

Builds on: SafeEraser MLLMU-Bench LLaVA NPO Prompt Decoupling

Closest papers by meaning

• Assessing and Learning Alignment of Unimodal Vision and Language Models (CVPR 2025 · similarity 0.89)
• ETA: Evaluating Then Aligning Safety of Vision Language Models at Inference Time (ICLR 2025 · similarity 0.89)
• Analyzing Fine-Grained Alignment and Enhancing Vision Understanding in Multimodal Language Models (NeurIPS 2025 · similarity 0.88)
• Fine-Grained Verifiers: Preference Modeling as Next-token Prediction in Vision-Language Alignment (ICLR 2025 · similarity 0.88)
• Unified Knowledge Maintenance Pruning and Progressive Recovery with Weight Recalling for Large Vision-Language Models (AAAI 2025 · similarity 0.88)
• Cross-Modal Safety Mechanism Transfer in Large Vision-Language Models (ICLR 2025 · similarity 0.87)