Research map / Label-efficient and robust learning / Recognition and long-tail
Spanning the Spectrum of Hatred Detection: A Persian Multi-Label Hate Speech Dataset with Annotator Rationales
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces PHATE, a manually annotated Persian multi-label hate speech dataset of ~7,000 tweets with target and rationale spans, and benchmarks ParsBERT, mBERT, XLM-R and ChatGPT, plus a rationale-assisted fine-tuning method.
Problem
Hate speech detection resources are scarce for low-resource languages such as Persian; existing Persian datasets rely on keyword-based collection and lack target and rationale annotations.
Approach
A Persian Twitter hate speech dataset collected without explicit slur search, with hierarchical multi-label sub-labels (violence, hate, vulgar), identity targets, and annotator rationale spans, plus a rationale-based auxiliary loss for fine-tuning.
Key results
- ParsBERT, mBERT and XLM-R reach about 78% F1 for hate speech detection; XLM-R is best on hate speech (78.1) and ParsBERT on vulgar.
- Zero-shot ChatGPT performs below fine-tuned models on all labels, with 64.3 F1 for hate speech and 36.2 F1 for violence.
- Rationale-assisted fine-tuning improves ParsBERT F1 on Violence (52.3 to 55.0) and Hate (60.8 to 63.3), but not Vulgar (60.3 to 60.3).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add rationale-assisted fine-tuning (ablation) | ParsBERT · PHATE test — Violence | F1: 52.3±2 -> 55.0±3 | Helped |
| add rationale-assisted fine-tuning (ablation) | ParsBERT · PHATE test — Hate | F1: 60.8±1 -> 63.3±1 | Helped |
| add rationale-assisted fine-tuning (ablation) | ParsBERT · PHATE test — Vulgar | F1: 60.3±1 -> 60.3±1 | No clear effect |
| mask annotated rationale spans instead of randomly masking the same number of words (ablation) | ParsBERT · PHATE test — Violence | F1: 46.2±1 -> 10.4±2 | Hurt |
| mask annotated rationale spans instead of randomly masking the same number of words (ablation) | ParsBERT · PHATE test — Hate | F1: 56.4±1 -> 25.5±2 | Hurt |
| mask annotated rationale spans instead of randomly masking the same number of words (ablation) | ParsBERT · PHATE test — Vulgar | F1: 56.3±1 -> 32.0±5 | Hurt |
Surprising findings
- Rationale-masked inputs cause a large F1 drop relative to random masking (e.g., Violence 46.2 to 10.4), supporting rationale validity.
- Zero-shot ChatGPT lags behind smaller fine-tuned encoders on every sub-label.
Limitations
- Data collected from a single platform (Twitter) and may not generalize to other platforms or contexts.
- Results are confined to Persian and tied to Persian keywords and cultural nuances.
- Annotation subjectivity may introduce biases; human performance on hate is moderate (F1 51.8).
Experimental setup
Datasets: PHATE
Benchmarks: PHATE
Metrics: precision recall F1
Baselines: ParsBERT mBERT XLM-R ChatGPT
Builds on: HateXplain Zampieri et al. 2019 hierarchical offensive labeling BERT
Closest papers by meaning
- A Multi-Task Benchmark for Abusive Language Detection in Low-Resource Settings (NeurIPS 2025 · similarity 0.87)
- Multi-Modal Discussion Transformer: Integrating Text, Images and Graph Transformers to Detect Hate Speech on Social Media (AAAI 2024 · similarity 0.87)
- TRACE: Textual Relevance Augmentation and Contextual Encoding for Multimodal Hate Detection (AAAI 2026 · similarity 0.86)
- IndicCONAN: A Multilingual Dataset for Combating Hate Speech in Indian Context (AAAI 2024 · similarity 0.86)
- Hateful Meme Detection through Context-Sensitive Prompting and Fine-Grained Labeling (Student Abstract) (AAAI 2025 · similarity 0.84)
- BeyondGender: A Multifaceted Bilingual Dataset for Practical Sexism Detection (AAAI 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.