Research map / Large language models / Agents, code and math
CARE-Bench: A Benchmark of Diverse Client Simulations Guided by Expert Principles for Evaluating LLMs in Psychological Counseling
Bichen Wang, Yixin Sun, Junzhe Wang, Hao Yang, Xing Fu, Yanyan Zhao, Si Wei, Shijin Wang, Bing Qin
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add expert-guided principles to the profile-based client simulation (ablation) | LLM client simulator (Qwen2.5-Max) · Human evaluation: Authenticity | 7-point Likert score: +0.40 *** | Improved |
| add expert-guided principles to the profile-based client simulation (ablation) | LLM client simulator (Qwen2.5-Max) · Human evaluation: Stayed in Role | 7-point Likert score: +0.23 ** | Improved |
| add expert-guided principles to the profile-based client simulation (ablation) | LLM client simulator (Qwen2.5-Max) · Human evaluation: Resembled Typical Case | 7-point Likert score: +0.36 *** | Improved |
| add expert-guided principles to the profile-based client simulation (ablation) | LLM client simulator (Qwen2.5-Max) · Human evaluation: Challenged the Counselor | 7-point Likert score: +0.25 ** | Improved |
| add expert-guided principles to the profile-based client simulation (ablation) | LLM client simulator (Qwen2.5-Max) · Human evaluation: Ready as Training Partner | 7-point Likert score: +0.23 ** | Improved |
| add expert-guided principles to the profile-based client simulation (ablation) | LLM client simulator (Qwen2.5-Max) · Human evaluation: Recommend to Novices | 7-point Likert score: +0.31 *** | Improved |
| use a reasoning model as counselor (Deepseek-R1 vs GPT-4o) | counselor LLM · CARE-Bench (CCS-R) | CCS-R avg: 3.81 -> 4.39 | Improved |
| scale Llama-3.1 from 8B to 70B | counselor LLM · CARE-Bench (WAI) | WAI avg: 3.51 -> 3.59 | Improved |
| use counseling-specialized model (CPsyCounX) vs GPT-4o | counselor LLM · CARE-Bench (BLRI) | BLRI avg: 1.59 -> 1.68 | Improved |
| client has low openness, conscientiousness, agreeableness or high neuroticism | counselor LLMs · CARE-Bench | counseling score | Hurt |
Closest papers by meaning
- TherapyGym: Evaluating and Aligning Clinical Fidelity and Safety in Therapy Chatbots (NeurIPS 2026 · similarity 0.88)
- Position: Beyond Assistance – Reimagining LLMs as Ethical and Adaptive Co-Creators in Mental Health Care (ICML 2025 · similarity 0.87)
- MedJourney: Benchmark and Evaluation of Large Language Models over Patient Clinical Journey (NeurIPS 2024 · similarity 0.86)
- Position: Medical Large Language Model Benchmarks Should Prioritize Construct Validity (ICML 2025 · similarity 0.85)
- An LLM-based Simulation Framework for Embodied Conversational Agents in Psychological Counseling (AAAI 2026 · similarity 0.85)
- Evaluating LLMs Across Multi-Cognitive Levels: From Medical Knowledge Mastery to Scenario-Based Problem Solving (ICML 2025 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
