Research map / Detection and segmentation / Open-set and few-shot detection
Towards Comprehensive Detection of Chinese Harmful Memes
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Builds ToxiCN MM, a 12,000-sample Chinese harmful meme dataset with harmful-type and modality-combination labels, and proposes MKE, a detector that adds LLM-generated text and image captions to a fine-tuned fusion model.
Problem
Detecting harmful memes on the Chinese Internet is under-studied because no reliable Chinese dataset or effective detector exists, and existing definitions focus on targeted harm and miss general offense, sexual innuendo, and dispirited culture.
Approach
Extend the harmful meme definition to include non-targeted potential toxicity, annotate fine-grained harmful types and text-image combinations, and enhance a multimodal classifier with GPT-4-generated captions of the inline text and image for contextual cultural knowledge.
Key results
- MKE with CLIP backbone reaches 80.23 macro F1 on harmful meme detection, versus 79.54 for CLIP alone.
- MKE (Fusion backbone) reaches 62.17 macro F1 on harmful type identification, versus 59.35 for the Fusion baseline.
- Zero-shot GPT-4 reaches 68.01 macro F1 on binary detection, below fine-tuned RoBERTa (76.36) and CLIP-based models.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add LLM-generated text and image captions with cross-attention | CLIP · ToxiCN MM test — harmful meme detection | macro F1: 79.54 -> 80.23 | Helped |
| Add LLM-generated text and image captions with cross-attention | Fusion (RoBERTa + ViT) · ToxiCN MM test — harmful meme detection | macro F1: 78.39 -> 79.16 | Helped |
| Add LLM-generated text and image captions with cross-attention | CLIP · ToxiCN MM test — harmful type identification | macro F1: 57.85 -> 61.47 | Helped |
| Add LLM-generated text and image captions with cross-attention | Fusion (RoBERTa + ViT) · ToxiCN MM test — harmful type identification | macro F1: 59.35 -> 62.17 | Helped |
| Add enhanced inline-text caption only (ablation) | Fusion (RoBERTa + ViT) · ToxiCN MM test — harmful meme detection | macro F1: 78.39 -> 79.04 | Helped |
| Add enhanced image caption only (ablation) | Fusion (RoBERTa + ViT) · ToxiCN MM test — harmful meme detection | macro F1: 78.39 -> 78.55 | Helped |
| Add enhanced inline-text caption only (ablation) | CLIP · ToxiCN MM test — harmful meme detection | macro F1: 79.54 -> 79.89 | Helped |
| Add enhanced image caption only (ablation) | CLIP · ToxiCN MM test — harmful meme detection | macro F1: 79.54 -> 80.07 | Helped |
| Add enhanced inline-text caption only (ablation) | Fusion (RoBERTa + ViT) · ToxiCN MM test — harmful type identification | macro F1: 59.35 -> 61.28 | Helped |
| Add enhanced image caption only (ablation) | Fusion (RoBERTa + ViT) · ToxiCN MM test — harmful type identification | macro F1: 59.35 -> 59.61 | Helped |
| Add enhanced inline-text caption only (ablation) | CLIP · ToxiCN MM test — harmful type identification | macro F1: 57.85 -> 59.23 | Helped |
| Add enhanced image caption only (ablation) | CLIP · ToxiCN MM test — harmful type identification | macro F1: 57.85 -> 59.24 | Helped |
Surprising findings
- Zero-shot GPT-4 underperforms fine-tuned pre-trained models, and adding the image input decreases its performance on general offense and dispirited culture memes.
- Over 50% of targeted harmful memes derive their toxicity from the combination of benign text and benign image.
Limitations
- Sampling was biased toward harmful content, and the dataset does not cover all harmful types such as fake news.
- Possible label noise from annotator subjectivity; Fleiss kappa for binary harmfulness is only 0.62.
- Study only predicts harmfulness and does not evaluate explanation generation.
Experimental setup
Datasets: ToxiCN MM
Benchmarks: ToxiCN MM
Metrics: precision recall macro F1 per-class F1
Baselines: GPT-3.5 GPT-4 RoBERTa ResNet ViT CLIP Fusion (RoBERTa+ViT) VisualGLM Qwen-VL VisCPM
Builds on: CLIP RoBERTa ViT
Closest papers by meaning
- T2Vs Meet VLMs: A Scalable Multimodal Dataset for Visual Harmfulness Recognition (NeurIPS 2024 · similarity 0.82)
- Efficient Toxic Content Detection by Bootstrapping and Distilling Large Language Models (AAAI 2024 · similarity 0.82)
- Hateful Meme Detection through Context-Sensitive Prompting and Fine-Grained Labeling (Student Abstract) (AAAI 2025 · similarity 0.82)
- TRACE: Textual Relevance Augmentation and Contextual Encoding for Multimodal Hate Detection (AAAI 2026 · similarity 0.82)
- Multi-Granular Multimodal Clue Fusion for Meme Understanding (AAAI 2025 · similarity 0.81)
- A Multi-Task Benchmark for Abusive Language Detection in Low-Resource Settings (NeurIPS 2025 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.