Research map / Large language models / AI and society
Towards Evaluating Proactive Risk Awareness of Multimodal Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces PaSBench, a 416-sample multimodal benchmark (128 image sequences, 288 text logs across 5 everyday safety domains) to test whether models proactively detect risks without user queries, finding low repeated-trial robustness despite decent knowledge.
Problem
Existing safety evaluations of LLMs and MLLMs are reactive, assuming users already know the risk and ask about it; there is no benchmark testing whether models proactively observe behavior/environment logs or images and warn users of everyday risks before harm occurs.
Approach
Builds a proactive risk-detection benchmark from safety knowledge points, generating image sequences (GPT-4o / GPT-4o-image) and text logs (DeepSeek-R1) stopping before the incident, and scores models with Best/Average/Worst-of-16 detection rates judged by GPT-4.1, then diagnoses failures with reactive and knowledge probes.
Key results
- Gemini-2.5-pro reaches 71% accuracy on the image set and 64% on the text set, yet misses 45% of image risks and 55% of text risks in repeated trials.
- Models of all sizes show high best-of-N potential; GPT-4.1-nano detects 87.5% of image risks across 32 runs.
- Models answer over 80% of knowledge multiple-choice questions correctly, and 68–93% of failed proactive cases are detected under the reactive setting, pointing to unstable proactive reasoning rather than knowledge gaps.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use a larger model rather than a smaller model within a model family | Model families compared in PaSBench · PaSBench image and text sets | accuracy, robustness, and potential | Helped |
| Replace proactive detection with a reactive prompt that supplies the safety knowledge and asks whether it is violated | Gemini-2.5-pro · Failed PaSBench cases, image and text | risk detections: Image: 552/596; Text: 1217/1646 | Helped |
| Replace proactive detection with a reactive prompt that supplies the safety knowledge and asks whether it is violated | GPT-4.1-nano · Failed PaSBench cases, image and text | risk detections: Image: 1047/1393; Text: 2525/3698 | Helped |
| Count nuanced answers as evidence that the model has the safety knowledge | Gemini-2.5-pro · PaSBench safety knowledge QA set | accuracy: 87% to 94.5% | Helped |
| Increase the number of sampling attempts | GPT-4.1-nano · PaSBench risk coverage | risk coverage: 87.5% across 32 runs; 91.4% across 128 runs | Helped |
| Increase observation length within the tested ranges | Evaluated models · PaSBench text and image sets | model performance | No clear effect |
Surprising findings
- Large reasoning models such as o1 performed notably worse than some non-reasoning models in the proactive setting.
- Non-reasoning Gemini-2.0-pro achieved the best overall performance, suggesting reasoning ability is not the current bottleneck.
Limitations
- Image samples contain only 1–4 sub-images and logs 4–8 observations, so the length range is limited and may not reflect longer sequences.
- The benchmark does not cover risk severity classification or appropriate response types, so models may over-report minor risks.
- Data is generated with GPT-4o and DeepSeek-R1 and English-only, with limited realism guarantees.
Experimental setup
Datasets: SafeText RESPONSE HealthBench
Benchmarks: PaSBench
Metrics: accuracy (average-of-16) potential (best-of-16) robustness (worst-of-16)
Baselines: Gemini-2.5-pro GPT-4.1-nano Qwen2.5-VL-7B o1 Claude-3.5-sonnet Llama-4-Maverick Llava-1.6-34B
Builds on: MSSBench LabSafetyBench GRPO
Closest papers by meaning
- Multimodal Situational Safety (ICLR 2025 · similarity 0.90)
- VLSU: Mapping the Limits of Joint Multimodal Understanding for AI Safety (ICLR 2026 · similarity 0.87)
- PROACT-Agent: Progressive Runtime Oversight and Active Circuit-breaking for Real-Time Safety (NeurIPS 2026 · similarity 0.86)
- When Safe Unimodal Inputs Collide: Optimizing Reasoning Chains for Cross-Modal Safety in Multimodal Large Language Models (AAAI 2026 · similarity 0.86)
- ProAct: A Benchmark and Multimodal Framework for Structure-Aware Proactive Response (ICML 2026 · similarity 0.86)
- RiOSWorld: Benchmarking the Risk of Multimodal Computer-Use Agents (NeurIPS 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.