Research map / Multi-agent systems and games / LLM agents
MDD-5k: A New Diagnostic Conversation Dataset for Mental Disorders Synthesized via Neuro-Symbolic LLM Agents
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes a neuro-symbolic multi-agent LLM framework that turns 1000 anonymized real patient cases into 5000 Chinese mental-disorder diagnostic conversations (MDD-5k) guided by a dynamic diagnosis tree.
Problem
Real psychiatrist-patient diagnostic conversations are nearly impossible to collect due to privacy and ethics, and directly prompting LLMs yields poor diagnostic dialogues, so labeled Chinese diagnostic conversation data is scarce.
Approach
One-to-many generation: each anonymized patient case yields multiple diverse conversations via randomized doctor personas, LLM-generated fictitious patient experiences from knowledge graphs, and random topic order, all controlled by a symbolic dynamic diagnosis tree (fixed symptom tree from SCID-5 plus dynamically parsed experience tree) managed by a tool agent.
Key results
- MDD-5k scored highest on professionalism (8.6), communication (8.3/8.4), and similarity (8.8) in 1-10 human ratings, versus D4 (6.6 professionalism, 7.2 similarity).
- MDD-5k dialogues average 6906.8 Chinese characters and 26.8 turns, roughly ten times longer than the compared datasets (D4 776 characters, 21.6 turns).
- Direct role-playing with gpt-4o scored notably lower on fluency (6.9 and 5.5) than the multi-agent framework's MDD-5k (8.6 and 7.6).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| neuro-symbolic multi-agent framework with dynamic diagnosis tree (vs direct role-playing gpt-4o) (ablation) | diagnostic conversation synthesis · Human evaluation | Professionalism / Similarity / Fluency (ii): Professionalism 8.6 vs 6.8; Similarity 8.8 vs 6.4; Fluency (ii) 7.6 vs 5.5 | Helped |
| direct role-playing LLM generation instead of multi-agent framework (ablation) | diagnostic conversation synthesis · Human evaluation | Fluency (i): 6.9 vs 8.6 for MDD-5k | Hurt |
| MDD-5k dataset (synthesized) vs D^4 real-psychiatrist dataset | dataset quality · Human evaluation | Fluency (i): 8.6 vs 8.6 | No clear effect |
| MDD-5k vs D^4 and CPsyCounD on human-rated quality | dataset quality · Human evaluation | Professionalism, communication, similarity: Professionalism 8.6 vs 6.6 (D^4); Similarity 8.8 vs 7.2 (D^4) | Helped |
| Psychological counseling dataset CPsyCounD used as diagnosis baseline | dataset quality · Human evaluation | Professionalism / Similarity: Professionalism 5.2; Similarity 4.4 | Hurt |
Surprising findings
- Psychiatrists judged synthesized patient responses as too 'sound and perfect' compared with real patients who struggle to answer.
- MDD-5k fluency (second criterion) was rated below D4, with the authors acknowledging repetitive content.
Limitations
- Synthesized conversations diverge from real diagnostics, with redundant symptom inquiries and patient responses that are too idealized.
- Dynamic diagnosis trees cover only depression, anxiety, sleep disorders, childhood emotional disorder and unspecified mood disorder; OCD and others are poorly addressed.
- Only a Chinese version is provided and the dataset is not yet released pending ethics review.
Experimental setup
Datasets: 1000 anonymized real patient cases from Shanghai Mental Health Center
Benchmarks: MDD-5k (human evaluation vs D4, CPsyCounD, direct role-playing)
Metrics: professionalism communication fluency similarity safety dialogue statistics
Baselines: D4 CPsyCounD Direct role-playing with gpt-4o
Builds on: SCID-5 ICD-10 CPsyCounD D4
Closest papers by meaning
- From Medical Records to Diagnostic Dialogues: A Clinical-Grounded Approach and Dataset for Psychiatric Comorbidity (ICLR 2026 · similarity 0.89)
- MoodAngels: A Retrieval-augmented Multi-agent Framework for Psychiatry Diagnosis (NeurIPS 2025 · similarity 0.87)
- Zhongjing: Enhancing the Chinese Medical Capabilities of Large Language Model through Expert Feedback and Real-World Multi-Turn Dialogue (AAAI 2024 · similarity 0.85)
- Thousand Voices of Trauma: A Large-Scale Synthetic Dataset for Modeling Prolonged Exposure Therapy Conversations (NeurIPS 2025 · similarity 0.85)
- MedExAgent: Training LLM Agents to Ask, Examine, and Diagnose in Noisy Clinical Environments (NeurIPS 2026 · similarity 0.85)
- AgentMental: An Interactive Multi-Agent Framework for Explainable and Adaptive Mental Health Assessment (AAAI 2026 · similarity 0.84)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.