Research map / Large language models / Language, safety and interpretability
PolyGuard: Massive Multi-Domain Safety Policy-Grounded Guardrail Dataset
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces PolyGuard, a 100k+ instance multi-domain guardrail benchmark grounded in 150+ safety policies across eight domains, and evaluates 19 guardrail models on it.
Problem
Existing guardrail benchmarks use ad hoc risk taxonomies that lack grounding in standardized safety policies and overlook domain-specific risks, limiting alignment with real-world operational requirements.
Approach
Extract a two-level risk hierarchy (risk categories and atomic safety rules) from official domain policies via GPT-4o, then generate rule-conditioned unsafe examples with uncensored LLMs and paired safe examples via detoxification prompting, plus multi-format and adversarially optimized (PAIR/AutoDAN) variants.
Key results
- Granite Guardian (5B) reaches the highest average attack success rate-based vulnerability with ASR of 92.8% averaged across domains on adversarial instances, while WildGuard shows the lowest average ASR at 19.8%.
- LlamaGuard average F1 on Instagram rises from 0.294 (LlamaGuard 1) to 0.605 (LlamaGuard 4), but Hate Speech F1 drops from 0.777 (LlamaGuard 3) to 0.734 (LlamaGuard 4).
- LlamaGuard 3 (1B) achieves higher average F1 than LlamaGuard 3 (8B) (0.485 vs. 0.423).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Evolve from LlamaGuard 1 to LlamaGuard 4 | LlamaGuard series · Instagram | Average category F1: 0.294 -> 0.605 | Helped |
| Evolve from LlamaGuard 1 to LlamaGuard 4 | LlamaGuard series · Instagram, Hate Speech category | F1: 0.777 (LlamaGuard 3) -> 0.734 (LlamaGuard 4) | Mixed |
| Scale from 1B to 8B | LlamaGuard 3 · Poly-Guard domains | Average F1: 0.485 vs. 0.423 | Hurt |
| Scale from 3B to 5B | Granite Guardian · Poly-Guard domains | Average F1: 0.774 vs. 0.749 | Hurt |
| Use conversational instances rather than single-statement or instruction-only instances | Guardrail models · Code and Cyber | F1: 12 out of 14 in the Code domain and 13 out of 14 in the Cyber domain; average improvement of over 5% | Helped |
| Append optimized adversarial suffixes to unsafe examples | Guardrail models · Attack-enhanced Poly-Guard | Attack success rate: average ASR exceeding 60% for Aegis Defensive and LLM Guard, and over 90% for Granite Guardian (5B) | Hurt |
| Use WildGuard rather than the other evaluated advanced guardrails | Guardrail models · Attack-enhanced Poly-Guard | Average attack success rate: 0.198 | Helped |
| Evaluate on high-severity rather than suggestive, lower-severity risk categories | Guardrail models · General Regulation (EU AI Act) | Attack success rate | Helped |
| Moderate Identity Misrepresentation rather than Hate Speech | Guardrail models · Instagram | F1: 0.715 -> 0.273 | Hurt |
| Increase model size from Granite Guardian (3B) to Granite Guardian (5B) | Granite Guardian · Finance and Law | F1 | Mixed |
| Evolve through the LlamaGuard series | LlamaGuard series · Instagram, common risk categories | F1: 0.777 -> 0.734 | Mixed |
| Evaluate on conversational rather than non-conversational instances | Guardrail models · Code and Cyber | F1: average improvement of over 5% in F1 score | Helped |
Surprising findings
- Newer versions of a guardrail series can lose performance on common risk categories even as coverage broadens.
- Smaller guardrails (LlamaGuard 3 1B, Granite Guardian 3B) sometimes outperform larger sibling models.
Limitations
- Policies are mostly sourced from Western institutions and global platforms, lacking culturally diverse and region-specific risks.
- Relies on discrete moderation outputs rather than likelihood-based metrics like AUPRC because APIs don't expose scores.
- Data generated by GPT-4o and o4-mini and uncensored LLMs may carry generator biases.
Experimental setup
Datasets: Official safety policy documents (150+) CWE Top 25 OWASP Top 10 MITRE ATT&CK CVE cases
Benchmarks: PolyGuard
Metrics: F1 recall false positive rate attack success rate
Baselines: LlamaGuard 1 LlamaGuard 2 LlamaGuard 3 (1B) LlamaGuard 3 (8B) LlamaGuard 4 ShieldGemma (2B) ShieldGemma (9B) TextMod API OmniMod API MDJudge 1 MDJudge 2 WildGuard
Builds on: PAIR AutoDAN WildGuard Llama Guard
Closest papers by meaning
- Position: Building Guardrails for Large Language Models Requires Systematic Design (ICML 2024 · similarity 0.88)
- DynaGuard: A Dynamic Guardian Model With User-Defined Policies (ICLR 2026 · similarity 0.88)
- SafeWatch: An Efficient Safety-Policy Following Video Guardrail Model with Transparent Explanations (ICLR 2025 · similarity 0.88)
- RSafe: Incentivizing proactive reasoning to build robust and adaptive LLM safeguards (NeurIPS 2025 · similarity 0.88)
- WildGuard: Open One-stop Moderation Tools for Safety Risks, Jailbreaks, and Refusals of LLMs (NeurIPS 2024 · similarity 0.88)
- On Calibration of LLM-based Guard Models for Reliable Content Moderation (ICLR 2025 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.