Research map / Large language models / Language, safety and interpretability
SafeDialBench: A Fine-Grained Safety Evaluation Benchmark for Large Language Models in Multi-Turn Dialogues with Diverse Jailbreak Attacks
Hongye Cao, Sijia Jing, Yanming Wang, Ziyue Peng, Zhixin Bai, Zhe Cao, Meng Fang, Fan Feng, Jiaheng Liu, Boyan Wang, Tianpei Yang, Jing Huo and 5 more
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| mix three assistant LLMs for dialogue generation instead of a single model (ablation) | SafeDialBench data construction · dataset diversity, 600 dialogues | entropy: 14.20-14.53 (single) -> 14.58 (mixed) | Improved |
| use the fallacy attack or purpose reverse jailbreak instead of topic change or reference attack (ablation) | multi-turn safety evaluation of top LLMs · SafeDialBench aggression and morality | safety score | Hurt |
| increase dialogue turns under fallacy attack (ablation) | evaluated LLMs · SafeDialBench six dimensions | safety score | Hurt |
| scale model from 7B to 14B (ablation) | Qwen-Chat · SafeDialBench handling ability | safety score | Improved |
| scale model from 7B to 13B (ablation) | Baichuan2-Chat · SafeDialBench | safety score | Mixed |
| use reasoning models (o3-mini, QwQ-32B, DeepSeek-R1) rather than chat models | LLM safety under jailbreak · SafeDialBench | safety score | Mixed |
Closest papers by meaning
- Multi-Turn Jailbreaking Large Language Models via Attention Shifting (AAAI 2025 · similarity 0.92)
- Speak Easy: Eliciting Harmful Jailbreaks from LLMs with Simple Interactions (ICML 2025 · similarity 0.91)
- MT-JailBench: A Modular Benchmark for Understanding Multi-Turn Jailbreak Attacks (NeurIPS 2026 · similarity 0.90)
- JailbreakBench: An Open Robustness Benchmark for Jailbreaking Large Language Models (NeurIPS 2024 · similarity 0.90)
- WildTeaming at Scale: From In-the-Wild Jailbreaks to (Adversarially) Safer Language Models (NeurIPS 2024 · similarity 0.89)
- SafePTR: Token-Level Jailbreak Defense in Multimodal LLMs via Prune-then-Restore Mechanism (NeurIPS 2025 · similarity 0.89)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
