Research map / Large language models / Language, safety and interpretability

LLMGuard: Guarding against Unsafe LLM Behavior

AAAI 2024 · Large language models › Language, safety and interpretability · approach: Safety alignment of LLMs · 14 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Presents LLMGuard, a post-processing tool that screens user prompts and LLM responses with an ensemble of five detectors (racial bias, violence, blacklisted topics, PII, toxicity) and blocks flagged content, demoed on FLAN-T5 and GPT-2.

Problem

LLMs in enterprise settings can generate inappropriate, biased, toxic, or privacy-leaking content, and retraining-based alignment is costly, so post-hoc guardrails on inputs and outputs are needed.

Approach

A modular library of independent specialized detectors wraps any LLM, passing every user prompt and LLM response through the ensemble and blocking the transaction if any detector flags unsafe content; blacklisted topics are configurable per-topic classifiers.

Key results

• Racial bias detector (LSTM) reaches 87.2% accuracy and 85.47% F1 on the test set.
• Violence detector (MLP over count vectors) reaches 86.4% accuracy on Jigsaw Toxicity 2021.
• Blacklisted-topic BERT classifiers achieve about 92% average accuracy; Detoxify reports mean AUC 98.64% on the Toxic Comment Classification Challenge 2018.

Limitations

• Evaluation is limited to per-detector classification metrics; no end-to-end evaluation of guarding LLM outputs is reported.
• The demo is qualitative and only covers two small LLMs (FLAN-T5 and GPT-2).
• PII detection relies on regular expressions with a reported NER F1 of 85%, with no further detail.

Experimental setup

Datasets: Twitter Texts Dataset Jigsaw Toxicity Dataset 2021 20-NewsGroup Dataset Wikipedia Comments Dataset Toxic Comment Classification Challenge 2018

Benchmarks: Twitter Texts Dataset Jigsaw Toxicity Dataset 2021 20-NewsGroup Dataset Toxic Comment Classification Challenge 2018

Metrics: accuracy F1 AUC NER F1

Builds on: Detoxify BERT LSTM

Closest papers by meaning

• WildGuard: Open One-stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMs (NeurIPS 2024 · similarity 0.86)
• BingoGuard: LLM Content Moderation Tools with Risk Levels (ICLR 2025 · similarity 0.85)
• From Judgment to Interference: Early Stopping LLM Harmful Outputs via Streaming Content Monitoring (NeurIPS 2025 · similarity 0.85)
• Look Before You Leap: Enhance Attention and Vigilance Regarding Harmful Content with GuidelineLLM (AAAI 2025 · similarity 0.85)
• On Calibration of LLM-based Guard Models for Reliable Content Moderation (ICLR 2025 · similarity 0.85)
• LLMScan: Causal Scan for LLM Misbehavior Detection (ICML 2025 · similarity 0.85)