Research map / Large language models / Language, safety and interpretability

MLLMGuard: A Multi-dimensional Safety Evaluation Suite for Multimodal Large Language Models

NeurIPS 2024 · Large language models › Language, safety and interpretability · approach: Safety alignment of LLMs · 7 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces MLLMGuard, a bilingual (English/Chinese) red-teaming safety benchmark for MLLMs across five dimensions, plus GuardRank, a lightweight trained evaluator, and evaluates 13 MLLMs.

Problem

Existing safety benchmarks for multimodal LLMs have limited dimension coverage, risk data leakage from open-source datasets, rely on costly human or biased GPT-4V judging for open-ended responses, and are almost entirely English.

Approach

A highly adversarial, human-curated bilingual image-text benchmark with 10+ red-teaming techniques adapted to MLLMs (82% social-media images to avoid leakage), scored on a 4-label Aware/Safe rubric, and judged by GuardRank, a fine-tuned LLaMA-2 / RoBERTa evaluator that outperforms GPT-4 as judge.

Key results

• GuardRank reaches 78.50% average accuracy on the validation set and 77.08% on the test set, versus 29.38% (zero-shot) and 42.78% (ICL) for GPT-4 on the test set.
• GPT-4V has the lowest average ASD (25.32), with MiniGPT-v2 close behind (ASD 27.03 on the averaged column per dimension) and the highest average PAR is MiniGPT-v2 at 49.72%.
• Most MLLMs are prone to hallucination on Non-existent Query and are strongly influenced by option position in Position Swapping, while 'Noise Injection' is largely defended against except by MiniGPT-v2.

Measured effects reported in the paper

Surprising findings

• GPT-4 as a zero-shot evaluator reaches only 29.38% accuracy, showing GPT-4 is unreliable as a safety judge.
• MiniGPT-v2 is among the safest on privacy, toxicity, and legality but performs poorly on truthfulness, suggesting a possible honesty-harmlessness trade-off.
• Scaling model parameters does not significantly improve MLLM safety and sometimes reduces it.

Limitations

• Only 1,500 of 2,282 samples are publicly released; the rest are held out to limit leakage, so results on the full set cannot be independently reproduced.
• GuardRank is trained on human-annotated scores for a limited set of model outputs and validated on responses from only a few models, so generalization is uncertain.
• Some evaluated models (e.g., Gemini, GPT-4V) are closed-source, and the appendix has inconsistencies such as the 'Safety Measures' rule for refusals.

Experimental setup

Datasets: Hateful Memes MEME Logo-2K+ Animal-10

Benchmarks: MLLMGuard

Metrics: ASD PAR accuracy

Baselines: GPT-4 zero-shot judge GPT-4 in-context judge GPT-4V Gemini

Builds on: LLaMA-2 RoBERTa MM-SafetyBench RTVLM

Closest papers by meaning

• SDEval: Safety Dynamic Evaluation for Multimodal Large Language Models (AAAI 2026 · similarity 0.93)
• Do We Really Need Curated Malicious Data for Safety Alignment in Multi-modal Large Language Models? (CVPR 2025 · similarity 0.92)
• MultiTrust: A Comprehensive Benchmark Towards Trustworthy Multimodal Large Language Models (NeurIPS 2024 · similarity 0.91)
• Multimodal Situational Safety (ICLR 2025 · similarity 0.91)
• SafetyPrompts: A Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safety (AAAI 2025 · similarity 0.91)
• When Understanding Becomes a Risk: Authenticity and Safety Risks in the Emerging Image Generation Paradigm (CVPR 2026 · similarity 0.90)