Research map / Vision-language and multimodal / Multimodal LLMs

SafeGRPO: Self-Rewarded Multimodal Safety Alignment via Rule-Governed Policy Optimization

CVPR 2026 · Vision-language and multimodal › Multimodal LLMs · approach: Multimodal LLM benchmarks · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes SafeGRPO, a GRPO-based multimodal safety alignment method with rule-governed self-rewards (format gate, modality safety-tag reward, refusal-behavior reward) trained on the 3K-pair SafeTag-VL-3K dataset, applied to Qwen3-VL Thinking models.

Problem

Multimodal LLMs have compositional safety risks from cross-modal text-image interactions, and reasoning-based safety training is unregulated; GRPO lacks verifiable signals for safety reasoning.

Approach

Integrates a rule-governed reward (format gate times averaged tag reward and keyword-based refusal behavior reward) into GRPO, with step-guided prompting that forces modality-level visual/textual/combined safety tags before the answer, so safety reasoning is verifiable without preference models.

Key results

• SafeGRPO on Qwen3-VL-8B-Thinking reaches 99.02 average jailbreak-defense safety score vs 97.69 for Think-in-Safety and 89.28 base.
• On SIUO, SafeGRPO (8B) achieves 94.31 safety score vs 86.52 base; MOSSBench refusal rate drops to 20.00 from 21.00 base while VLGuard rises to 95.00.
• On general benchmarks, SafeGRPO (8B) average 78.75 vs 77.98 base, while Think-in-Safety drops to 52.02 and VLGuard to 16.94.

Surprising findings

• Training-based SFT baselines (VLGuard, Think-in-Safety) raise over-refusal sharply on MOSSBench (e.g., 95% refusal for VLGuard on 8B), whereas the RL method slightly lowers it.
• SafeGRPO slightly improves general capability benchmarks over the base model rather than degrading them.

Limitations

• Training set is small (3K pairs) and partially GPT-5 annotated; annotation reliability checked only by informal volunteer inspection with no reported agreement metric.
• Behavior reward uses keyword matching for refusal detection, which is brittle.
• Safety evaluation relies on GPT-4o-mini as judge.

Experimental setup

Datasets: SafeTag-VL-3K VLGuard SPA-VL BeaverTails

Benchmarks: FigStep VLGuard test set MM-SafetyBench SIUO MOSSBench ScienceQA IconQA MathVista MM-Vet POPE

Metrics: Safety Score (0-100) Refusal Rate accuracy

Baselines: VLGuard ECSO Think-in-Safety

Builds on: GRPO VLGuard Think-in-Safety verl Qwen3-VL

Closest papers by meaning

• When Safe Unimodal Inputs Collide: Optimizing Reasoning Chains for Cross-Modal Safety in Multimodal Large Language Models (AAAI 2026 · similarity 0.92)
• AlphaAlign: Incentivizing Safety Alignment with Extremely Simplified Reinforcement Learning (ICLR 2026 · similarity 0.91)
• Safe RLHF-V: Safe Reinforcement Learning from Multi-modal Human Feedback (NeurIPS 2025 · similarity 0.90)
• Towards Safe Reasoning in Large Reasoning Models via Corrective Intervention (ICLR 2026 · similarity 0.90)
• Pragma-VL: Towards a Pragmatic Arbitration of Safety and Helpfulness in MLLMs (ICLR 2026 · similarity 0.90)
• Evolving Contextual Safety in Multi-Modal Large Language Models via Inference-Time Self-Reflective Memory (CVPR 2026 · similarity 0.90)