Research map / Image generation / Text-to-image diffusion
Hate in Plain Sight: On the Risks of Moderating AI-Generated Hateful Illusions
Yiting Qu, Ziqing Yang, Yihan Ma, Michael Backes, Savvas Zannettou, Yang Zhang
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply Gaussian blur followed by histogram equalization before moderation (ablation) | GPT-4o · Hateful Illusion | detection accuracy: 0.03 -> 0.70 | Improved |
| apply Gaussian blur followed by histogram equalization before moderation (ablation) | Q16 · Hateful Illusion | detection accuracy: 0.21 -> 0.82 | Improved |
| use chain-of-thought prompting instead of zero-shot for VLM moderation (ablation) | VLMs · Hateful Illusion | detection accuracy: best 0.034 -> 0.102 (hate speech) | Mixed |
| full fine-tuning of CLIP backbone for prompt learning (ablation) | CLIP prompt-learning classifier · Hateful Illusion test split | accuracy: 0.637 -> 0.938 | Improved |
| full fine-tuning of CLIP backbone for linear probing (ablation) | CLIP linear probe · Hateful Illusion test split | accuracy: 0.622 -> 0.823 | Improved |
| hide hate messages inside optical illusions rather than showing them directly | Q16 classifier · Hate speech | detection accuracy: 0.696 -> 0.209 | Hurt |
| add optical-illusion cues to zero-shot VLM prompts (ablation) | VLMs · Hateful Illusion | detection accuracy | No clear effect |
| increase ControlNet guidance scale (ablation) | hateful illusion generation · Hateful illusion generation | percentage of hateful illusions | Improved |
| re-enable Stable Diffusion safety checker during generation | Stable Diffusion · generated hateful illusions | share of images blocked: blocks 3.0% | No clear effect |
Closest papers by meaning
- Semi-Truths: A Large-Scale Dataset of AI-Augmented Images for Evaluating Robustness of AI-Generated Image detectors (NeurIPS 2024 · similarity 0.85)
- Debate with Images: Detecting Deceptive Behaviors in Multimodal Large Language Models (ICML 2026 · similarity 0.85)
- TRACE: Textual Relevance Augmentation and Contextual Encoding for Multimodal Hate Detection (AAAI 2026 · similarity 0.85)
- Analyzing and Mitigating Object Hallucination: A Training Bias Perspective (AAAI 2026 · similarity 0.84)
- GHOST: Hallucination-Inducing Image Generation for Multimodal LLMs (ICLR 2026 · similarity 0.84)
- Image Safeguarding: Reasoning with Conditional Vision Language Model and Obfuscating Unsafe Content Counterfactually (AAAI 2024 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
