Research map / Vision-language and multimodal / Human and embodied modeling
Image Safeguarding: Reasoning with Conditional Vision Language Model and Obfuscating Unsafe Content Counterfactually
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes ConditionalBLIP, an InstructBLIP-based VLM conditioned on unsafe-image classifiers for rationale generation, and CSE, a counterfactual region-obfuscation method using Bayesian adaptive superpixels (BASS) and FullGrad attribution with greedy search to minimally blur unsafe image regions.
Problem
Platforms must both explain why an image is unsafe (sexual, cyberbullying, self-harm content) and obfuscate only the minimal unsafe regions so that safe regions remain visible for moderation and investigation; existing VLMs lack conditioning on domain-specific unsafe attributes and region-based coun
Approach
Condition a Q-Former-based instruction-tuned VLM on the control code of pre-trained unsafe-image classifiers to ground rationales in unsafe attributes, and generate minimal obfuscation masks by combining FullGrad attribution-guided Bayesian superpixel segmentation (BASS) with an informed greedy region search for a counterfactual that flips the classifier output.
Key results
- ConditionalBLIP reached human-evaluated coarse-grained accuracy of 94.5% (sexually explicit), 99.5% (cyberbullying), and 99.0% (self-harm), versus InstructBLIP at 80.0%, 23.0%, and 47.0%.
- BASS segmentation with FullGrad achieved 81.8% successful counterfactual examples, search depth 5.5, and about 40.1% of the image obfuscated on average.
- CSE outperformed CSRA by 13.9% on sexually explicit, 22.0% on cyberbullying, and 39.5% on self-harm images in counterfactual success.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Condition unsafe-image reasoning on classifier guidance through CIIT | ConditionalBLIP · Sexually Explicit, Cyberbullying, and Self-Harm | human evaluation: 94.5 vs 80.0; 99.5 vs 23.0; 99.0 vs 51.5 | Helped |
| Condition image reasoning on classifier guidance through CIIT | ConditionalBLIP vs InstructBLIP · Fine-grained moderator questions across Sexually Explicit, Cyberbullying, and Self-Harm | detection accuracy: 99.2 vs 61.1; 99.9 vs 37.9; 80.0 vs 59.1; 94.0 vs 99.2; 98.3 vs 56.4; 95.2 vs 57.2; 98.9 vs 27.4; 99.2 vs 74.2; 81.4 vs | Mixed |
| Replace BASS segmentation with SLIC (ablation) | CSE · Sexually Explicit, Cyberbullying, and Self-Harm | successful counterfactual examples: 90.6 -> 76.6; 82.0 -> 60.0; 72.8 -> 33.4 | Hurt |
| Replace BASS segmentation with Felzenswalb, Watershed, or SAM (ablation) | CSE · Sexually Explicit, Cyberbullying, and Self-Harm | successful counterfactual examples: 19.9, 51.2, 29.5; 20.5, 50.0, 50.0; 38.4, 33.1, 39.5 | Hurt |
| Replace FullGrad with Ablation-CAM, Grad-CAM, Grad-CAM++, or XGrad-CAM attribution (ablation) | CSE with BASS segmentation · Cyberbullying | successful counterfactual examples: 82.0 -> 79.5 | Hurt |
| Replace FullGrad attribution with another tested attribution map (ablation) | CSE with BASS segmentation · Sexually Explicit and Self-Harm | successful counterfactual examples: 90.6; 72.8 | No clear effect |
| Set unsafe classifier guidance on CIIT instruct prompt embedding from 1 to 0 (ablation) | ConditionalBLIP · Unsafe-image rationale generation | rationale specificity | Hurt |
Surprising findings
- Attribution method choice (FullGrad vs Grad-CAM variants) changed counterfactual success only marginally, with FullGrad slightly better.
- Felzenszwalb and SAM segmentation yielded much lower counterfactual success than BASS (e.g., 19.9% and 29.5% vs 90.6% on sexually explicit images).
Limitations
- Evaluation of rationales relies on human evaluators from the research team using majority voting, with no standardized benchmark.
- Counterfactual search is limited to a fixed region budget (10 regions) rather than exhaustive search.
Experimental setup
Datasets: NSFW images dataset (334,327 images) cyberbullying images dataset (~20,000 images) self-harm images dataset (5,000 images) ImageNet
Metrics: human-evaluated categorization accuracy fine-grained question accuracy counterfactual success rate average search depth average obfuscation percentage
Baselines: InstructBLIP mPLUG OFA-Large CSRA SLIC Felzenszwalb Compact Watershed SAM
Builds on: InstructBLIP BLIP-2 FullGrad Bayesian adaptive superpixel segmentation (BASS) CSRA
Closest papers by meaning
- MLLM-as-a-Judge for Image Safety without Human Labeling (CVPR 2025 · similarity 0.88)
- Towards Policy-Adaptive Image Guardrail: Benchmark and Method (CVPR 2026 · similarity 0.87)
- VLSU: Mapping the Limits of Joint Multimodal Understanding for AI Safety (ICLR 2026 · similarity 0.86)
- Video-SafetyBench: A Benchmark for Safety Evaluation of Video LVLMs (NeurIPS 2025 · similarity 0.86)
- Safe + Safe = Unsafe? Exploring How Safe Images Can Be Exploited to Jailbreak Large Vision-Language Models (NeurIPS 2025 · similarity 0.86)
- GuardTrace-VL: Detecting Unsafe Multimodel Reasoning via Iterative Safety Supervision (CVPR 2026 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.