Research map / Large language models / AI and society
Position: Building Guardrails for Large Language Models Requires Systematic Design
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Position paper arguing that LLM guardrails should be built through a systematic, socio-technical, neural-symbolic and verification-driven process, reviewing Llama Guard, NeMo Guardrails and Guardrails AI.
Problem
Existing open-source LLM guardrails (Llama Guard, Nvidia NeMo Guardrails, Guardrails AI) are ad hoc, loosely coupled, and lack guarantees and systematic requirement handling for conflicting, multi-category safety requirements.
Approach
Argues guardrails should be engineered like safety-critical software: socio-technical requirement elicitation with multidisciplinary teams, deeper neural-symbolic coupling, and a V-model lifecycle with certified verification, Pareto-based trade-off evaluation and safety arguments.
Key results
- Position paper with no new experimental results; reviews three open-source guardrails and classifies them as Type-1 (Llama Guard, NeMo) and Type-2 (Guardrails AI) neural-symbolic systems.
- Proposes covering requirements in four areas: unintended responses, fairness, privacy and copyright, and hallucination/uncertainty, with a V-model style development and verification process.
Surprising findings
- Argues that defenses that block known attacks remain vulnerable to stronger attacks, creating an endless attack-defense cycle.
- Notes that guardrail adoption can make ChatGPT responses more conservative, with character counts on sensitive queries dropping from over 600 to about 140.
Limitations
- No new experiments or quantitative evaluation of the proposed systematic design approach.
- Scope limited to three open-source guardrails; closed-source solutions excluded.
Experimental setup
Baselines: Llama Guard NeMo Guardrails Guardrails AI
Builds on: Llama Guard NeMo Guardrails Guardrails AI randomized smoothing
Closest papers by meaning
- PolyGuard: Massive Multi-Domain Safety Policy-Grounded Guardrail Dataset (NeurIPS 2025 · similarity 0.88)
- On Evaluating the Durability of Safeguards for Open-Weight LLMs (ICLR 2025 · similarity 0.88)
- On Calibration of LLM-based Guard Models for Reliable Content Moderation (ICLR 2025 · similarity 0.88)
- RigorLLM: Resilient Guardrails for Large Language Models against Undesired Content (ICML 2024 · similarity 0.86)
- DynaGuard: A Dynamic Guardian Model With User-Defined Policies (ICLR 2026 · similarity 0.86)
- MoGU: A Framework for Enhancing Safety of LLMs While Preserving Their Usability (NeurIPS 2024 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.