Research map / Adversarial robustness and security / Attacks on text and graphs
Safeguarding Vision-Language Models: Mitigating Vulnerabilities to Gaussian Noise in Perturbation-based Attacks
Jiawei Wang, Yushen Zuo, Yuanjun Chai, Zhendong Liu, Yicheng Fu, Yichun Feng, Kin-Man Lam
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add Gaussian noise (sigma 0.1) to input images | InternVL2-8B · MM-Vet | performance score: 59.9 -> 54.4 | Hurt |
| add Gaussian noise to input images | MiniGPT-4-13B · RealToxicityPrompts | attack success rate: 34.8 -> 44.1 | Hurt |
| add Gaussian noise to input images | LLaVA-v1.5-7B · RealToxicityPrompts | attack success rate: 57.7 -> 60.1 | Hurt |
| fine-tune with Robust-VLGuard and noise augmentation instead of VLGuard | MiniGPT-4-13B · RealToxicityPrompts (noisy) | attack success rate: 43.7 -> 16.5 | Improved |
| fine-tune with Robust-VLGuard instead of VLGuard | InternVL2-8B · MM-Vet (clean) | performance score: 42.9 -> 56.2 | Improved |
| fine-tune with Robust-VLGuard | LLaVA-v1.5-7B · RealToxicityPrompts (noisy) | attack success rate: 60.1 -> 42.3 | Improved |
| fine-tune with Robust-VLGuard | InternVL2-8B · RealToxicityPrompts (noisy) | attack success rate: 57.2 -> 34.5 | Improved |
| increase proportion of safety data in instruction ratio (4:1 to 4:4) (ablation) | Robust-VLGuard fine-tuning InternVL2-8B · MM-Vet / RealToxicityPrompts | attack success rate | Mixed |
| increase fine-tuning epochs from 1 to 4 (ablation) | Robust-VLGuard fine-tuning InternVL2-8B · MM-Vet / RealToxicityPrompts | attack success rate | Mixed |
| apply DiffPure (t*=50) to Gaussian-noisy images without noise-augmented fine-tuning (ablation) | MiniGPT-4-13B · RealToxicityPrompts | attack success rate: 44.1 -> 44.3 | No clear effect |
| apply DiffPure (t*=50) to adversarial images (eps=64/255) (ablation) | MiniGPT-4-13B · RealToxicityPrompts | attack success rate: 67.2 -> 44.5 | Improved |
| use DiffPure instead of JailGuard preprocessing (ablation) | LLaVA-RobustVLGuard · RealToxicityPrompts (eps=32/255) | attack success rate: 48.9 -> 43.9 | Improved |
Closest papers by meaning
- Safety Fine-Tuning at (Almost) No Cost: A Baseline for Vision Large Language Models (ICML 2024 · similarity 0.92)
- Attention! Your Vision Language Model Could Be Maliciously Manipulated (NeurIPS 2025 · similarity 0.92)
- Exploring Visual Vulnerabilities via Multi-Loss Adversarial Search for Jailbreaking Vision-Language Models (CVPR 2025 · similarity 0.92)
- Retention Score: Quantifying Jailbreak Risks for Vision Language Models (AAAI 2025 · similarity 0.91)
- ETA: Evaluating Then Aligning Safety of Vision Language Models at Inference Time (ICLR 2025 · similarity 0.91)
- JailBound: Jailbreaking Internal Safety Boundaries of Vision-Language Models (NeurIPS 2025 · similarity 0.91)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
