Research map / Large language models / Language, safety and interpretability
RTP-LX: Can LLMs Evaluate Toxicity in Multilingual Scenarios?
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces RTP-LX, a human-transcreated and human-annotated 28-language toxicity corpus, and evaluates 10 small/large language models and Azure Content Safety as multilingual toxicity annotators, finding high accuracy but low agreement with humans.
Problem
Whether LLMs and SLMs can reliably annotate toxic content in culturally-sensitive multilingual settings, given that accuracy-based evaluation may be misleading.
Approach
A participatory-design multilingual toxicity benchmark built from transcreated RTP prompts plus manually curated culturally-specific prompts, with multi-category ordinal human labels, used to measure S/LLM agreement with humans via weighted Cohen's kappa rather than accuracy alone.
Key results
- Llama Guard had the highest percentage agreement on toxic prompts, closely followed by Gemma 7B and GPT-4 Turbo, but dropped to near last in weighted kappa.
- GPT-4 Turbo was significantly better than other models in weighted kappa and came closest to human judgements (0.22 average IRR difference).
- All models decreased in weighted kappa from high- to low-resource languages, by up to around 10%; Gemma 2B had up to 40% false positives on benign completions while Llama Guard and ACS had near-zero.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use GPT-4 Turbo rather than other evaluated S/LLMs | Multilingual toxicity annotation on RTP-LX toxic prompts · RTP-LX toxic prompts | weighted Cohen's kappa | Helped |
| Use Llama Guard rather than other S/LLMs | RTP-LX toxic prompts · RTP-LX toxic prompts | percentage agreement | Helped |
| Use ACS rather than the evaluated S/LLM approaches | Multilingual toxicity detection on RTP-LX · RTP-LX toxic prompts | percentage agreement | Helped |
| Use a newer Llama model version (ablation) | Llama models on RTP-LX · RTP-LX toxic prompts | agreement with human annotators | Helped |
| Increase model size within Llama 2 or Llama 3 (ablation) | Llama models on RTP-LX · RTP-LX toxic prompts | agreement with human annotators | No clear effect |
| Use Mistral-v3 rather than Mistral-v2 (ablation) | Mistral models on RTP-LX · RTP-LX | weighted Cohen's kappa | Helped |
| Evaluate on lower-resource rather than higher-resource languages (ablation) | All evaluated S/LLMs · RTP-LX prompts subset | weighted Cohen's kappa: up to around 10\% | Hurt |
| Evaluate on culturally specific manual prompts rather than transcreated prompts (ablation) | FLORES Toxicity-200 · RTP-LX manual versus transcreated subset | block rate: -8\% average | Hurt |
| Use S/LLMs to detect microaggressions, bias, or identity attacks | Evaluated S/LLMs · RTP-LX toxic prompts | agreement with human annotators | Hurt |
| Use Gemma 2B for toxicity classification | Gemma 2B · RTP-LX benign completions | false-positive rate: up to 40\% | Hurt |
| Use Llama Guard for toxicity classification | Llama Guard · RTP-LX benign completions | false-positive rate: near-zero | Helped |
Surprising findings
- Llama Guard topped percentage agreement yet ranked near the bottom in weighted kappa, showing accuracy is misleading due to class imbalance.
- Models failed to detect subtle harms such as microaggressions, bias, and identity attacks, and tended to output binary or higher-valued labels.
Limitations
- Corpus has US-centric skew from RTP and limited dialectal coverage, mostly Indo-European languages.
- Only base, un-finetuned models were evaluated; newer model versions may perform differently.
- Risk of data contamination since the corpus could be used in future training, though data is password-protected.
Experimental setup
Datasets: RTP (Real Toxicity Prompts) FLORES Toxicity-200
Benchmarks: RTP-LX
Metrics: percentage agreement weighted Cohen's kappa exact-match block rate false positive rate
Baselines: Azure Content Safety FLORES Toxicity-200 block list Llama Guard
Builds on: RTP (Real Toxicity Prompts) Azure Content Safety
Closest papers by meaning
- SafetyPrompts: A Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safety (AAAI 2025 · similarity 0.92)
- SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal (ICLR 2025 · similarity 0.88)
- MLLMGuard: A Multi-dimensional Safety Evaluation Suite for Multimodal Large Language Models (NeurIPS 2024 · similarity 0.88)
- Multilingual Jailbreak Challenges in Large Language Models (ICLR 2024 · similarity 0.88)
- SafeWorld: Geo-Diverse Safety Alignment (NeurIPS 2024 · similarity 0.87)
- Efficient Toxic Content Detection by Bootstrapping and Distilling Large Language Models (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.