Research map / Vision-language and multimodal / Multimodal LLMs

Is Your Multimodal Language Model Oversensitive to Safe Queries?

ICLR 2025 · Vision-language and multimodal › Multimodal LLMs · approach: Benchmarks and analysis of MLLMs · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces MOSSBench, a 300-sample multimodal benchmark showing that 20 MLLMs refuse benign queries paired with visual stimuli (exaggerated risk, negated harm, counterintuitive interpretation), and that safer models are more oversensitive.

Problem

MLLMs often refuse harmless multimodal queries when images contain superficially alarming elements, and oversensitivity (false refusals) has been under-studied compared with harmful-compliance safety evaluation, especially for multimodal models.

Approach

Defines three visual stimulus types that trigger oversensitivity (exaggerated risk, negated harm, counterintuitive interpretation), builds a benign VQA benchmark via LLM-generated scenarios plus human filtering and AMT validation, and traces refusals to perception, intent reasoning, and safety judgement stages using oracle captions and intents.

Key results

• Claude 3 opus (web version) refuses 76.33% of benign queries under human evaluation, and Gemini Advanced 63.67%.
• GPT-4o (API) and GPT-4V refuse under 10% of benign queries, while Claude 3 sonnet API reaches 65.33% human-evaluated refusal.
• Safer models are more oversensitive: Gemini-Pro 1.5 refusal rises from 29.33% to 45.33% when given a safety-focused system prompt.

Measured effects reported in the paper

Surprising findings

• Web versions of proprietary models refuse far more than their API counterparts (Claude 3 opus 30.67% API vs 76.33% web).
• Oversensitivity is strongest for Negated Harm and Counterintuitive Interpretation stimuli, with errors often arising at the final safety judgement stage even when oracle captions and intents are provided.

Limitations

• Benchmark covers only a small fraction of possible oversensitivity scenarios (300 samples).
• Scenario generation still relies on human filtering, limiting scalability.
• Web-version results are manual and hard to verify due to proprietary filters and undisclosed safety mechanisms.

Experimental setup

Datasets: MOSSBench

Benchmarks: MOSSBench

Metrics: refusal rate average refusal rate

Baselines: GPT-4o GPT-4V Gemini-Pro 1.5 Gemini-Pro Vision Claude 3 opus Claude 3 sonnet Claude 3 haiku Reka IDEFICS-9b-Instruct Qwen-VL-Chat InternLM-XComposer2-7b InstructBLIP-Vicuna-7b

Builds on: Chain-of-thought safety analysis framework (stage decomposition) HarmBench

Closest papers by meaning

• Multimodal Situational Safety (ICLR 2025 · similarity 0.89)
• Safety Geometry Collapse in Multimodal LLMs and Adaptive Drift Correction (NeurIPS 2026 · similarity 0.87)
• MLLMGuard: A Multi-dimensional Safety Evaluation Suite for Multimodal Large Language Models (NeurIPS 2024 · similarity 0.87)
• Dictionary-Aligned Concept Control for Safeguarding Multimodal LLMs (CVPR 2026 · similarity 0.87)
• Do We Really Need Curated Malicious Data for Safety Alignment in Multi-modal Large Language Models? (CVPR 2025 · similarity 0.87)
• MLLMs Fail to Refuse when Using Tools Agentically (NeurIPS 2026 · similarity 0.86)