Research map / Vision-language and multimodal / Multimodal LLMs
TRUST-VLM: Thorough Red-Teaming for Uncovering Safety Threats in Vision-Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes TRUST-VLM, an automatic black-box red-teaming framework that uses an LLM red-teamer with in-context learning, text-to-image generation, and feedback-driven refinement to produce diverse, effective, low-toxicity multimodal test cases for VLMs.
Problem
Red-teaming VLMs for safety failures is underexplored; existing methods have low fault detection rates, limited diversity, and produce overtly toxic or easily detected test cases.
Approach
Generates paired text prompts and images with a red-teaming LLM via in-context learning, executes them on the target VLM, judges outputs with moderation models, and iteratively refines failed cases using feedback from a zero-shot classifier that maps VLM responses and moderation analyses to rewriting instructions.
Key results
- TRUST-VLM achieves 95% fault detection rate on LLaVA-v1.5-13B versus 72% for JailBreakV-28K and 24% for RTVLM.
- Average fault detection rate of 82.04% on GPT-4o over 200 rounds.
- Achieves textual diversity 0.88 and textual alignment 0.76, with lower visual toxicity (51%) than HADES (99%).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use TRUST-VLM instead of RTVLM | LLaVA-v1.5-13B red-teaming · LLaVA-v1.5-13B | fault detection rate: 24.00% -> 95.00% | Helped |
| use TRUST-VLM instead of JailBreakV-28K | LLaVA-v1.5-13B red-teaming · LLaVA-v1.5-13B | fault detection rate: 72.00% -> 95.00% | Helped |
| use TRUST-VLM instead of RTVLM | LLaVA-v1.5-13B red-teaming · LLaVA-v1.5-13B | textual diversity: 0.55 -> 0.88 | Helped |
| use TRUST-VLM instead of RTVLM | LLaVA-v1.5-13B red-teaming · LLaVA-v1.5-13B | visual diversity: 0.52 -> 0.50 | Hurt |
| use TRUST-VLM instead of RTVLM | LLaVA-v1.5-13B red-teaming · LLaVA-v1.5-13B | visual toxicity: 58.17% -> 51.00% | Helped |
| use TRUST-VLM instead of JailBreakV-28K | LLaVA-v1.5-13B red-teaming · LLaVA-v1.5-13B | textual toxicity: 5.33% -> 11.67% | Hurt |
| use TRUST-VLM instead of HADES | red-teaming VLMs · reported comparison | fault detection rate: 100% -> 99% | Hurt |
| use TRUST-VLM instead of HADES | red-teaming VLMs · reported comparison | textual toxicity: 83.00% -> 12.00% | Helped |
| use TRUST-VLM instead of HADES | red-teaming VLMs · reported comparison | textual diversity: 0.91 -> 0.88 | Hurt |
| use 8 ICL examples instead of 1 (ablation) | TRUST-VLM · LLaVA-v1.5-13B red-teaming | textual diversity: 0.71 -> 0.88 | Helped |
| use 8 ICL examples instead of 1 (ablation) | TRUST-VLM · LLaVA-v1.5-13B red-teaming | textual toxicity: 19.00% -> 11.67% | Helped |
| use 8 ICL examples instead of 1 (ablation) | TRUST-VLM · LLaVA-v1.5-13B red-teaming | fault detection rate: 97% -> 95% | Hurt |
Surprising findings
- Using fewer ICL examples (1 vs 8) raises FDR slightly but increases toxicity and lowers diversity and alignment, showing a trade-off between exploitability and stealthiness.
Limitations
- High computational cost due to iterative refinement.
- Reliance on off-the-shelf safety-aligned models causes occasional refusals during tip and caption generation.
- Feedback mapping and refinement could be more fine-grained.
Experimental setup
Datasets: JailBreakV-28K RTVLM
Benchmarks: JailBreakV-28K RTVLM HADES Arondight
Metrics: fault detection rate diversity (1-AvgSelfBLEU, CLIP image diversity) alignment (CLIP/BERT cosine similarity) toxicity (detector ensembles)
Baselines: JailBreakV-28K RTVLM Arondight HADES
Builds on: in-context learning Perez et al. red-teaming with language models Arondight zero-shot classification with BART-Large-MNLI
Closest papers by meaning
- ReaLM: A Unified Red-Teaming Benchmark for Physical-World VLMs (NeurIPS 2026 · similarity 0.92)
- RedVLA: Physical Red Teaming for Vision-Language-Action Models (NeurIPS 2026 · similarity 0.89)
- ETA: Evaluating Then Aligning Safety of Vision Language Models at Inference Time (ICLR 2025 · similarity 0.88)
- Chain of Attack: On the Robustness of Vision-Language Models Against Transfer-Based Adversarial Attacks (CVPR 2025 · similarity 0.87)
- HarmBench: A Standardized Evaluation Framework for Automated Red Teaming and Robust Refusal (ICML 2024 · similarity 0.87)
- Pragma-VL: Towards a Pragmatic Arbitration of Safety and Helpfulness in MLLMs (ICLR 2026 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.