Research map / Large language models / Instruction and fine-tuning
UniDetox: Universal Detoxification of Large Language Models via Dataset Distillation
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes UniDetox, which distills detoxifying text from GPT-2 XL via contrastive decoding between base and toxic models, then fine-tunes diverse LLMs on that text to reduce toxicity without per-model tuning.
Problem
Existing LLM detoxification methods are model-specific, require per-model toxic models and hyperparameter tuning, and trade off toxicity against language modeling quality.
Approach
Use contrastive decoding between a base model and a toxic fine-tuned version as an efficient dataset distillation procedure to produce discrete detoxifying text; fine-tuning any LLM on this text approximates moving opposite the toxic vector, so one distilled set and one hyperparameter config transfer across models.
Key results
- On GPT-2 XL, UniDetox (lr=5e-5) reduces ID toxicity probability from 0.53 to 0.40 and OOD from 0.41 to 0.25, the lowest among compared methods, with PPL 10.38 vs 17.28 for base.
- Distilled from GPT-2 XL only, UniDetox (lr=5e-5) lowers LLaMA2-7B ID TP from 0.58 to 0.29 and OOD from 0.49 to 0.26 using a single shared configuration.
- Distilled text shows lower Detoxify score (0.003 vs 0.008) and reduced left-leaning political bias (44.56% vs 50.81%) compared with GPT-2 samples.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use contrastively distilled detoxifying text rather than GPT-2 sampled text (ablation) | GPT-2 XL · ToxiGen ID | TP: 0.48\textsubscript{ 0.02} -> 0.40\textsubscript{ 0.00 } | Helped |
| Use contrastively distilled detoxifying text rather than GPT-2 sampled text (ablation) | GPT-2 XL · ToxiGen OOD | TP: 0.35\textsubscript{ 0.03} -> 0.25\textsubscript{ 0.02 } | Helped |
| Set plausibility constraint alpha to 0.05 rather than 0.1 (ablation) | UniDetox on GPT-2 XL · ToxiGen ID | TP: 0.46\textsubscript{ 0.02 } -> 0.62\textsubscript{ 0.02 } | Hurt |
| Use learning rate 5e-5 rather than 1e-5 (ablation) | UniDetox on GPT-2 XL · ToxiGen ID | TP: 0.46\textsubscript{ 0.02 } -> 0.40\textsubscript{ 0.00 } | Helped |
| Apply GPT-2-distilled text without model-specific distillation or tuning | UniDetox on OPT-6.7B · ToxiGen ID | TP: 0.58\textsubscript{ 0.01} -> 0.28\textsubscript{ 0.00 } | Helped |
| Apply GPT-2-distilled text without model-specific distillation or tuning | UniDetox on Falcon-7B · ToxiGen ID | TP: 0.30\textsubscript{ 0.01 } -> 0.33\textsubscript{ 0.00 } | Mixed |
| Apply GPT-2-distilled text without model-specific distillation or tuning | UniDetox on LLaMA2-7B · ToxiGen ID | TP: 0.45\textsubscript{ 0.03} -> 0.29\textsubscript{ 0.01 } | Helped |
| Use the long safety preprompt | GPT-2 XL · ToxiGen ID | TP: 0.53\textsubscript{ 0.01} -> 0.63\textsubscript{ 0.01} | Hurt |
| Use contrastively distilled text rather than GPT-2 sampled text (ablation) | OPT-6.7B · ToxiGen ID | TP: 0.61\textsubscript{ 0.01} -> 0.28\textsubscript{ 0.00 } | Helped |
Surprising findings
- Text sampled directly from GPT-2 XL without contrastive decoding also detoxifies other models, sometimes outperforming Task Arithmetic or DExperts.
- The Safety Preprompt has no positive effect and the long version even worsens toxicity on GPT-2 XL.
Limitations
- Perplexity measured with LLaMA2-7B, and PPL/diversity trade-off reduces Dist-n at lr=5e-5.
- Gains are not uniform; UniDetox at lr=1e-5 is weaker on some models and on Falcon-7B DExperts performs best on TP under Perspective API.
- Detoxification evaluated mainly on ToxiGen with Detoxify; only one distillation base (GPT-2 XL) and a small set of target models.
Experimental setup
Datasets: DGHS ToxiGen MMLU
Benchmarks: ToxiGen MMLU
Metrics: Toxicity Probability Expected Maximum Toxicity perplexity Dist-1/2/3 MMLU accuracy
Baselines: Safety Preprompt GPT-2 Samples LM-Steer DExperts Task Arithmetic
Builds on: dataset distillation DExperts Task Arithmetic contrastive decoding adaptive plausibility constraint
Closest papers by meaning
- Large Language Models can Become Strong Self-Detoxifiers (ICLR 2025 · similarity 0.92)
- Detoxifying Large Language Models via Autoregressive Reward Guided Representation Editing (NeurIPS 2025 · similarity 0.91)
- IF-Guide: Influence Function-Guided Detoxification of LLMs (NeurIPS 2025 · similarity 0.87)
- MUSE: Machine Unlearning Six-Way Evaluation for Language Models (ICLR 2025 · similarity 0.87)
- Model Editing as a Robust and Denoised variant of DPO: A Case Study on Toxicity (ICLR 2025 · similarity 0.87)
- In-Context Unlearning: Language Models as Few-Shot Unlearners (ICML 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.