Research map / Large language models / Language, safety and interpretability
Efficient Toxic Content Detection by Bootstrapping and Distilling Large Language Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes BD-LLM, which bootstraps LLMs with a confidence-gated, context-tree prompting method (DToT) and distills DToT rationales into a small FastChat-T5 student for toxic content detection.
Problem
Toxic content detection with supervised LMs has limited accuracy and cross-dataset transfer, while LLM prompting is hard to design and too costly to deploy in production.
Approach
Decision-Tree-of-Thought (DToT) iteratively re-prompts an LLM with more fine-grained toxicity-category context when its confidence is low, selecting the child context via rationale relevance; the extracted answers and rationales are then distilled into a 3B student LM that predicts both label and rationale.
Key results
- DToT+FS+R raises ChatGPT accuracy on Toxigen to 87.03% vs 82.71% for CoT, and DToT improves FC-T5 accuracy by up to 5.30% over CoT.
- FC-T5 (3B) fine-tuned with labels and DToT rationales reaches 87.53% accuracy on Toxigen, versus 82.45% for fine-tuned RoBERTa.
- Student trained on Toxigen with DToT rationales achieves AUC 75.90 on SBIC and 77.31 on DHate, outperforming RoBERTa's 65.46 and 61.51 AUC on those datasets.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace CoT prompting with DToT prompting | ChatGPT · Toxigen | accuracy: 82.71 -> 85.03 | Helped |
| Replace CoT prompting with DToT prompting | ChatGPT · SBIC | accuracy: 68.50 -> 71.60 | Helped |
| Replace CoT prompting with DToT prompting | ChatGPT · DHate | accuracy: 65.20 -> 68.20 | Helped |
| Add few-shot demonstrations to DToT (ablation) | ChatGPT · Toxigen | accuracy: 85.03 -> 86.03 | Helped |
| Add rationale augmentations to DToT with few-shot demonstrations (ablation) | ChatGPT · Toxigen | accuracy: 86.03 -> 87.03 | Helped |
| Add few-shot demonstrations and rationale augmentations to DToT (ablation) | ChatGPT · DHate | accuracy: 68.20 -> 69.20 | Mixed |
| Add DToT-extracted rationales during fine-tuning (ablation) | FC-T5 with human labels · Toxigen | accuracy: 84.99 -> 87.53 | Helped |
| Use DToT-extracted rather than CoT-extracted rationales during fine-tuning (ablation) | FC-T5 with human labels · Toxigen | accuracy: 87.31 -> 87.53 | Helped |
| Add DToT-extracted rationales during fine-tuning (ablation) | FC-T5 with human labels · SBIC | accuracy: 84.00 -> 85.10 | Helped |
| Use DToT-extracted rather than CoT-extracted rationales during fine-tuning (ablation) | FC-T5 with human labels · DHate | accuracy: 86.20 -> 86.40 | Helped |
| Add DToT-extracted rationales during fine-tuning (ablation) | FC-T5 fine-tuned on Toxigen · Toxigen | accuracy: 84.99 -> 87.53 | Helped |
| Add DToT-extracted rationales during fine-tuning (ablation) | Flan-T5-XL 3B · Toxigen | accuracy: 85.54 -> 87.53 | Helped |
Surprising findings
- Fine-tuning FC-T5 with labels only makes it output Yes/No without rationales, while fine-tuning with DToT rationales restores rich rationale generation.
- Without rationale distillation, the fine-tuned FC-T5 does not outperform RoBERTa on Toxigen, but with DToT rationales and predicted teacher labels it does.
Limitations
- Context selector uses greedy search at each iteration, which may not be globally optimal.
- Context tree is predefined and cannot adapt dynamically to LLM responses.
- Only a two-level context tree and maximal two DToT steps were tested.
Experimental setup
Datasets: Toxigen SBIC DHate Amazon internal toxic dataset
Benchmarks: Toxigen SBIC DHate Amazon internal dataset
Metrics: accuracy F1 AUC
Baselines: RoBERTa fine-tuned CoT prompting UniLC FC-T5 with labels only FC-T5 with CoT rationales
Builds on: Tree-of-Thought chain-of-thought UniLC Flan-T5 SCOTT
Closest papers by meaning
- RTP-LX: Can LLMs Evaluate Toxicity in Multilingual Scenarios? (AAAI 2025 · similarity 0.87)
- BingoGuard: LLM Content Moderation Tools with Risk Levels (ICLR 2025 · similarity 0.87)
- Large Language Models can Become Strong Self-Detoxifiers (ICLR 2025 · similarity 0.87)
- Time Travel in LLMs: Tracing Data Contamination in Large Language Models (ICLR 2024 · similarity 0.86)
- A Multi-Task Benchmark for Abusive Language Detection in Low-Resource Settings (NeurIPS 2025 · similarity 0.86)
- SafetyPrompts: A Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safety (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.