Research map / Large language models / Language, safety and interpretability
What Are They Filtering Out? An Experimental Benchmark of Filtering Strategies for Harm Reduction in Pretraining Datasets
Marco Antonio Stranisci, Christian Hardmeier
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| apply rule-based and classifier-based toxicity filters | Common Crawl pretraining samples · named-entity mentions by gender and origin | mentions removed (%): Western women -4.2% (Shutterstock) vs Western men -2.3% | Hurt |
| apply quality-based (document similarity) filtering as a proxy for safety | Common Crawl pretraining samples · harmful sentence removal | toxic sentences remaining (%): 90.5% remain (Wikipedia), 69.3% remain (OpenWebText) | No clear effect |
| use Shutterstock vs HateBase lexicon (ablation) | rule-based filtering · Common Crawl samples | type of content removed | Mixed |
| use FastText vs profanity-check classifier (ablation) | toxicity classifier filtering · Common Crawl samples | overlap of flagged sentences: 35.8% of FastText-flagged also flagged by profanity-check | Mixed |
| use document-similarity filters (Wikipedia/OpenWebText classifiers) | Common Crawl pretraining samples · named-entity mentions by gender | mentions removed (%): -44.6% Western men vs -33.1% Western women (webtext) | Hurt |
Closest papers by meaning
- SafetyPrompts: A Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safety (AAAI 2025 · similarity 0.87)
- Underestimated Privacy Risks for Minority Populations in Large Language Model Unlearning (ICML 2025 · similarity 0.87)
- Learning and Forgetting Unsafe Examples in Large Language Models (ICML 2024 · similarity 0.86)
- BingoGuard: LLM Content Moderation Tools with Risk Levels (ICLR 2025 · similarity 0.85)
- SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal (ICLR 2025 · similarity 0.85)
- Task Contamination: Language Models May Not Be Few-Shot Anymore (AAAI 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
