Research map / Large language models / Language, safety and interpretability
How Catastrophic is Your LLM? Certifying Risk in Conversation
Chengxiao Wang, Isha Chaudhary, Qian Hu, Weitong Ruan, Rahul Gupta, Gagandeep Singh
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| constrain final query to harmful target set (graph path) vs vanilla last query (ablation) | graph path distribution · HarmBench cybercrime, GPT-4o judge | certified lower bound: 0.403 -> 0.533 (Mistral); 0.123 -> 0.195 (Claude) | Improved |
| use adaptive-with-rejection sampling instead of random node with jailbreak on safer LLMs (ablation) | gpt-oss / Claude · HarmBench chembio / cybercrime | certified lower bound: 0.028 -> 0.101 (gpt-oss chembio); 0.028 -> 0.195 (Claude cyber) | Improved |
| add jailbreak prefixes to random node queries (ablation) | Nova Premier / Claude · HarmBench | certified lower bound | No clear effect |
| increase jailbreak insertion probability (ablation) | DeepSeek / Mistral / Llama · HarmBench | certified lower bound | Improved |
| increase jailbreak probability (ablation) | Claude / gpt-oss · HarmBench | certified lower bound | No clear effect |
| increase query sequence length (ablation) | Llama / DeepSeek / Claude · HarmBench | certified lower bound | Improved |
| increase query-set size from 50 to 150 (ablation) | certification · HarmBench scenario | certified bounds | No clear effect |
| vary AwR weight ratio, graph thresholds (ablation) | certification · HarmBench scenario | certified bounds | No clear effect |
| certify over distribution instead of fixed single-turn evaluation | certified lower bound vs ST ASR · HarmBench cybercrime | number of specs exceeding baseline: 157 of specs (DeepSeek) vs ST; 22 vs MT | Improved |
| use certified distributions vs fixed multi-turn baseline | certified lower bound vs MT ASR · HarmBench chembio | number of specs exceeding baseline: 30 (Mistral), 29 (DeepSeek), 1 (Claude) | Improved |
Closest papers by meaning
- Navigating the Safety Landscape: Measuring Risks in Finetuning Large Language Models (NeurIPS 2024 · similarity 0.88)
- SafetyPrompts: A Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safety (AAAI 2025 · similarity 0.88)
- SG-Bench: Evaluating LLM Safety Generalization Across Diverse Tasks and Prompt Types (NeurIPS 2024 · similarity 0.87)
- ChatBug: A Common Vulnerability of Aligned LLMs Induced by Chat Templates (AAAI 2025 · similarity 0.87)
- OR-Bench: An Over-Refusal Benchmark for Large Language Models (ICML 2025 · similarity 0.87)
- Safety Layers in Aligned Large Language Models: The Key to LLM Security (ICLR 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
