Research map / Large language models / AI and society
Thousand Voices of Trauma: A Large-Scale Synthetic Dataset for Modeling Prolonged Exposure Therapy Conversations
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Thousand Voices of Trauma, a synthetic dataset of 3,000 Prolonged Exposure therapy conversations (500 cases x 6 phases) generated with Claude Sonnet 3.5, plus an emotional trajectory benchmark scoring models against a Claude baseline.
Problem
Trauma-focused therapy conversation data for AI development is scarce due to privacy and ethics constraints, and existing mental health datasets lack demographic diversity and PE-specific session structure.
Approach
Generate structured, demographically diverse synthetic PE therapy sessions across six protocol phases using rule-based and probabilistic profile sampling, validate with clinicians, and define an emotional-trajectory benchmark that compares model SUDS-like intensity ratings to a Claude Sonnet 3.5 baseline using Pearson, DTW, and RMSE.
Key results
- Mistral Large scored highest on the emotional trajectory benchmark (Pearson 0.80, DTW 2.38, RMSE 1.07, S_abs 0.74) against the Claude Sonnet 3.5 baseline.
- Mistral Small (S_abs 0.59) scored below the smaller Mistral 7B Instruct (S_abs 0.63), which the authors attribute to instruction-following rather than model size.
- Seven therapists rated two synthetic PE transcripts as detailed and engaging but only somewhat appropriate and of limited therapist skill level (novice to competent).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use Mistral Large rather than Mistral Small for emotional trajectory assessment | Models evaluated against the Claude Sonnet 3.5 baseline · Emotional trajectory benchmark | S_abs: 0.59 -> 0.74 | Helped |
| Use Mistral 7B Instruct rather than Mistral Small for emotional trajectory assessment | Mistral models evaluated against the Claude Sonnet 3.5 baseline · Emotional trajectory benchmark | S_abs: 0.59 -> 0.63 | Helped |
| Use Mistral Large rather than Mistral Small for emotional trajectory assessment | Models evaluated against the Claude Sonnet 3.5 baseline · Emotional trajectory benchmark | Pearson: 0.61 -> 0.80 | Helped |
| Use Mistral Large rather than Mistral Small for emotional trajectory assessment | Models evaluated against the Claude Sonnet 3.5 baseline · Emotional trajectory benchmark | RMSE: 1.70 -> 1.07 | Helped |
| Use instruction-tuned Mistral 7B rather than Mistral Small | Mistral models evaluated on the emotional trajectory benchmark · Emotional trajectory benchmark | Pearson: 0.61 -> 0.62 | Helped |
Surprising findings
- Mistral Small underperformed the smaller Mistral 7B Instruct on the trajectory benchmark, suggesting instruction adherence matters more than size.
Limitations
- Predefined categories and distributions may not capture real-world therapy complexity.
- Probabilistic generation adds randomness that may hurt reproducibility of specific analyses.
- Expert ratings suggest AI therapist behavior is closer to novice than expert and perceived value/appropriateness varied.
Experimental setup
Datasets: Synthetic PE therapy conversations generated by Claude Sonnet 3.5
Benchmarks: Thousand Voices of Trauma emotional trajectory benchmark
Metrics: Pearson correlation Dynamic Time Warping RMSE Absolute Composite Score
Baselines: Mistral Large Amazon Nova Pro Llama 3 70B Instruct Llama 3.1 70B Instruct Llama 3 8B Instruct Llama 3.1 8B Instruct Mistral 7B Instruct Mistral Small
Builds on: Prolonged Exposure therapy protocol
Closest papers by meaning
- TherapyGym: Evaluating and Aligning Clinical Fidelity and Safety in Therapy Chatbots (NeurIPS 2026 · similarity 0.87)
- From Medical Records to Diagnostic Dialogues: A Clinical-Grounded Approach and Dataset for Psychiatric Comorbidity (ICLR 2026 · similarity 0.86)
- An LLM-based Simulation Framework for Embodied Conversational Agents in Psychological Counseling (AAAI 2026 · similarity 0.85)
- MindGuard: Guardrail Classifiers for Multi-Turn Mental Health Support (NeurIPS 2026 · similarity 0.85)
- MDD-5k: A New Diagnostic Conversation Dataset for Mental Disorders Synthesized via Neuro-Symbolic LLM Agents (AAAI 2025 · similarity 0.85)
- Mental-Perceiver: Audio-Textual Multi-Modal Learning for Estimating Mental Disorders (AAAI 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.