Research map / Large language models / Language, safety and interpretability
CLINIC : Evaluating Multilingual Trustworthiness in Language Models for Healthcare
Akash Ghosh, Srivarshinee Sridhar, Raghav Kaushik Ravi, Muhsin Muhsin, Sriparna Saha, Chirag Agarwal
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| move from high- to low-resource language | Phi-4mini · Honesty | honesty rate: 83.50 -> 24.50 | Hurt |
| move from high- to low-resource language | GPT-4o-mini · Honesty | honesty rate: 78.38 -> 68.50 | Hurt |
| move from high- to low-resource language | Gemini-2.5-Pro · Honesty | honesty rate: 95.20 -> 93.00 | Hurt |
| move from high- to low-resource language | MMedLLama · Consistency | similarity: 0.657 -> 0.573 | Hurt |
| move from high- to low-resource language | DSeek-R1 · Consistency | similarity: 0.749 -> 0.680 | Hurt |
| move from high- to low-resource language | GPT-4o-mini · Consistency | similarity: 0.781 -> 0.743 | Hurt |
| use medical fine-tuned model instead of general model | OpenBioLLM-8B vs LLaMA-3.2-3B · Honesty | honesty rate: 75.50 -> 40.75 | Hurt |
| move from high- to mid-resource language | Gemini-2.5-Pro · OOD drug RtA | RtA: 90.87 -> 97.33 | Improved |
| move from high- to low-resource language | Qwen-2-1.5B · OOD drug RtA | RtA: 62.50 -> 41.67 | Hurt |
| move from high- to low-resource language | Qwen3-32B · Sycophancy | similarity score: 0.054 -> 0.018 | Hurt |
Closest papers by meaning
- MedJourney: Benchmark and Evaluation of Large Language Models over Patient Clinical Journey (NeurIPS 2024 · similarity 0.90)
- CARES: A Comprehensive Benchmark of Trustworthiness in Medical Vision Language Models (NeurIPS 2024 · similarity 0.88)
- Cross-Care: Assessing the Healthcare Implications of Pre-training Data on Language Model Bias (NeurIPS 2024 · similarity 0.88)
- Multilingual Medical Language Models: A Path to Improving Lay Health Worker Effectiveness (Student Abstract) (AAAI 2024 · similarity 0.88)
- MedAlign: A Clinician-Generated Dataset for Instruction Following with Electronic Medical Records (AAAI 2024 · similarity 0.87)
- Trustworthy and Practical AI for Healthcare: A Guided Deferral System with Large Language Models (AAAI 2025 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
