Research map / Vision-language and multimodal / Visual question answering

Watch Out Your Album! On the Inadvertent Privacy Memorization in Multi-Modal Large Language Models

ICML 2025 · Vision-language and multimodal › Visual question answering · approach: Multimodal LLM question answering · 1 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Shows that randomly generated task-irrelevant watermarks (synthetic usernames/user_ids) embedded in VQA fine-tuning images can be inadvertently memorized by MLLMs via mini-batch spurious correlations, detectable through layer-wise linear probing but not by direct prompting or MIA.

Problem

Whether MLLMs inadvertently memorize private content that is entirely irrelevant to the fine-tuning task, through spurious correlations arising in partial mini-batch training.

Approach

Embed random task-irrelevant private watermarks into VQA fine-tuning images and use gradient-similarity analysis plus layer-wise linear probes that distinguish seen from unseen watermarks to detect memorization, while varying embedding rate and batch size.

Key results

• Fine-tuning with embedded watermarks leaves downstream VQA performance largely unchanged (e.g., LLaVA-1.5 ScienceQA on COCO 70.0 vs 68.5 at r=1.0).
• Batch gradient cosine similarity drops from 98.3 (origin) to 92.9 (with privacy) for LLaVA-1.5 on COCO, comparable to image transformations (85.3).
• Probing accuracy for seen vs unseen usernames exceeds 90% after fine-tuning on VisualGenome for Qwen-VL, while direct prompting for user_id yields 0% accuracy; MIA AUC stays near 50 after tuning.

Measured effects reported in the paper

Surprising findings

• Direct prompting for the user_id yields 0% response accuracy even though linear probes on hidden states reach over 90% accuracy.
• MIA methods (LOSS, Zlib, Min-k% Prob) barely change after fine-tuning, failing to expose the watermarks.

Limitations

• Private content is synthetic (GPT-4 generated), not real user data, and only 5 or 100 items per subset were used.
• Probing results are limited to binary seen/unseen classification and the paper does not establish a model-specific safe batch size.
• Attack methods are shown to be weak; the paper does not show practical extraction of the memorized user_ids.

Experimental setup

Datasets: COCO GQA OCR-VQA TextVQA VisualGenome ScienceQA MME-Perception

Benchmarks: ScienceQA MME-Perception

Metrics: accuracy MME score average batch cosine gradient similarity probing accuracy AUC ROC

Baselines: Origin & Origin gradient baseline ImageTransf. TextTransf. LOSS Zlib Entropy Min-k% Prob

Builds on: LoRA LLaVA-1.5 Qwen-VL

Closest papers by meaning

• Extracting Training Data From Document-Based VQA Models (ICML 2024 · similarity 0.88)
• Black-Box Membership Inference Attack for LVLMs via Prior Knowledge-Calibrated Memory Probing (NeurIPS 2025 · similarity 0.87)
• Understanding Information Storage and Transfer in Multi-Modal Large Language Models (NeurIPS 2024 · similarity 0.86)
• MLLMs Know Where to Look: Training-free Perception of Small Visual Details with Multimodal LLMs (ICLR 2025 · similarity 0.86)
• Quantifying and Analyzing Entity-Level Memorization in Large Language Models (AAAI 2024 · similarity 0.86)
• Déjà Vu Memorization in Vision–Language Models (NeurIPS 2024 · similarity 0.86)