Research map / Large language models / Language, safety and interpretability
CARES: Comprehensive Evaluation of Safety and Adversarial Robustness in Medical LLMs
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces CARES, a medical LLM safety benchmark of ~18K prompts across 8 safety principles, 4 harm levels and direct/jailbreak rewrites, with a three-way Accept/Caution/Refuse protocol and Safety Score, plus a jailbreak-type classifier used for reminder-based mitigation.
Problem
Existing safety benchmarks for LLMs lack clinical specificity, graded harmfulness, jailbreak-style adversarial prompts, and measurement of over-refusal of benign prompts in healthcare settings.
Approach
Generate clinically grounded prompts across graded harm levels, rewrite each with three jailbreak strategies (indirect, obfuscation, role-play), evaluate responses with a three-way Accept/Caution/Refuse judge and a partial-credit Safety Score, and mitigate via a trained jailbreak-type classifier that supplies reminder prompts.
Key results
- O4-mini (SS 0.71) and DeepSeek-R1 (SS 0.70) achieve the highest Safety Scores among evaluated models.
- Models show significant Safety Score drops on jailbroken prompts versus direct prompts.
- Jailbreak-aware reminder conditioning gives substantial gains for vulnerable models such as Claude-3.5-Haiku and Llama-3.2-3B-Instruct (exact values not given in text).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Evaluate direct prompts against their jailbroken variants | evaluated language models · CARES safety test dataset | Safety Score | Hurt |
| Use indirect or role-play prompt variants instead of direct prompts | evaluated language models · CARES jailbreak prompting strategies | refusal rate | Hurt |
| Use jailbreak-aware reminders based on a classifier detecting the jailbreak type | evaluated language models · CARES safety evaluation | Safety Score | Helped |
| Use jailbreak-aware reminders based on a classifier detecting the jailbreak type | GPT-4o-mini and DeepSeek-V3 · CARES safety evaluation | Safety Score | Helped |
| Use jailbreak-aware reminders based on a classifier detecting the jailbreak type | Claude-3.5-Haiku, Llama-3.1-8B-Instruct, and Llama-3.2-3B-Instruct · CARES safety evaluation | Safety Score | Helped |
Surprising findings
- Harmless (level 0) and highly harmful (level 3) prompts are harder than mid-level prompts, attributed to jailbreak rewrites making safe prompts look suspicious.
- Medically tuned models such as Medalpaca-13B and Meditron-70B perform competitively with larger general models.
Limitations
- Only three jailbreak strategies (indirect, obfuscation, role-play) are covered; multi-turn and tool-use attacks are not.
- Prompts are synthetic and generated by LLMs, and GPT-4o-mini is used as judge; manual check covers only 200 samples.
- Reported classifier and mitigation gains are described qualitatively in the text.
Experimental setup
Datasets: CARES-18K
Benchmarks: CARES-18K 9K safety test split
Metrics: Safety Score Accuracy F1-score
Baselines: GPT-4o-mini GPT-4.1 O4-mini Claude-3.5-Haiku Claude-3.7-Sonnet DeepSeek-V3 DeepSeek-R1 Llama-3.1-70B
Builds on: MedSafetyBench Collective Constitutional AI
Closest papers by meaning
- MedSafetyBench: Evaluating and Improving the Medical Safety of Large Language Models (NeurIPS 2024 · similarity 0.89)
- SafetyPrompts: A Systematic Review of Open Datasets for Evaluating and Improving Large Language Model Safety (AAAI 2025 · similarity 0.88)
- SORRY-Bench: Systematically Evaluating Large Language Model Safety Refusal (ICLR 2025 · similarity 0.88)
- Does Safety Training of LLMs Generalize to Semantically Related Natural Prompts? (ICLR 2025 · similarity 0.88)
- Medical MLLM Is Vulnerable: Cross-Modality Jailbreak and Mismatched Attacks on Medical Multimodal Large Language Models (AAAI 2025 · similarity 0.88)
- CARES: A Comprehensive Benchmark of Trustworthiness in Medical Vision Language Models (NeurIPS 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.