Research map / Vision-language and multimodal / Multimodal LLMs

VLMGuard: Bootstrapping Malicious Prompt Detectors from Unlabeled Vision-Language Prompts in the Wild

NeurIPS 2026 · Vision-language and multimodal › Multimodal LLMs · approach: VLM capability analysis · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes VLMGuard, which trains a malicious prompt detector from unlabeled in-the-wild VLM prompts by scoring maliciousness via SVD projection energy of VLM hidden states, then training an MLP classifier on the resulting pseudo-partition.

Problem

Malicious prompt detection for vision-language models is hard because labeled benign and malicious prompt data are scarce, while unlabeled user prompts from deployment are abundant but mixed.

Approach

Estimate a per-prompt maliciousness score as variance-weighted projection energy onto the top-k singular directions of centered VLM embeddings of an unlabeled mixture (valid under rare contamination via a Huber model), then train a safeguarding MLP classifier on the noisy pseudo-partition to generalize beyond direct projection under prompt variation.

Key results

• VLMGuard with Qwen2.5-VL-7B reaches 94.79% average AUROC across three settings, versus 85.33% for the best LLM-based baseline (Qwen2.5-VL-72B) and 82.70% for ASTRA.
• With LLaVA-1.6-7B, VLMGuard averages 93.00% AUROC, above HiddenDetect (88.53%) and ASTRA (83.86%).
• On Qwen2.5-VL-72B, VLMGuard reaches 97.48% AUROC on VLGuard & MSSBench.

Surprising findings

• Direct SVD projection scores are less robust than the learned classifier; the classifier improves over direct thresholding on all three settings.
• Text-only VLMGuard with Qwen2.5-7B-Instruct reaches 79.65 AUROC, above its self-detection judge at 62.84.

Limitations

• Hyperparameters (layer, k, threshold) are selected on a small labeled validation set of 100 samples, which departs from fully unlabeled claims.
• Pseudo-label precision is low at small pi (25% at pi=0.005), so the classifier trains on noisy labels.
• Malicious labels rely on dataset-native labels, and the boundary of maliciousness is context-dependent.

Experimental setup

Datasets: JailBreakV-28K GPT4V-Caption VLGuard MLLMGuard MSSBench MMMU

Benchmarks: JailBreakV & GPT4V VLGuard & MLLMGuard VLGuard & MSSBench

Metrics: AUROC AUPRC

Baselines: Self-detection LlamaGuard3-Vision LLaVAGuard Ovis2-34B InternVL3-78B-Instruct Qwen2.5-VL-72B-Instruct GPT-5.4 JailGuard Perplexity GradSafe Gradient Cuff MirrorCheck

Builds on: Huber contamination model HiddenDetect ASTRA GradSafe

Closest papers by meaning

• Backdooring Vision-Language Models with Out-Of-Distribution Data (ICLR 2025 · similarity 0.88)
• One Prompt Word is Enough to Boost Adversarial Robustness for Pre-trained Vision-Language Models (CVPR 2024 · similarity 0.88)
• Attention! Your Vision Language Model Could Be Maliciously Manipulated (NeurIPS 2025 · similarity 0.87)
• Chain of Attack: On the Robustness of Vision-Language Models Against Transfer-Based Adversarial Attacks (CVPR 2025 · similarity 0.86)
• Retention Score: Quantifying Jailbreak Risks for Vision Language Models (AAAI 2025 · similarity 0.86)
• MMJ-Bench: A Comprehensive Study on Jailbreak Attacks and Defenses for Vision Language Models (AAAI 2025 · similarity 0.86)