AI Paper Atlas

Research map / Learning theory and algorithms / Fairness

Moving Beyond Medical Exams: A Clinician-Annotated Fairness Dataset of Real-World Tasks and Ambiguity in Mental Healthcare

Max Lamparth, Declan Grabb, Amy Franks, Scott Gershan, Kaitlyn N Kunstman, Aaron Lulla, Monika Drummond Roots, Manu Sharma, Aryan Shrivastava, Nina Vasan, Colleen Waickman

ICLR 2026 · Learning theory and algorithms › Fairness · approach: Fairness auditing and regularization

Read the paperCheck your idea against 137,217 papers

AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Measured effects reported in the paper

What was changedWhereMeasuredEffect
mental-health or clinical fine-tuning (MMedS-Llama-3-8B) vs base Llama 3.1 8B (ablation)MMedS-Llama-3-8B · MENTATaccuracy: 0.65 -> 0.58No clear effect
fine-tuned open-source health models vs Llama2/Llama3 counterparts (ablation)health fine-tuned LMs · MENTATaccuracyNo clear effect
use larger closed-source models vs smaller open-source modelsclosed-source LMs · MENTATaccuracy: Llama 3.1 8B 0.65 -> o1 0.81Improved
ambiguous categories (triage, documentation) vs unambiguous categoriesall LMs · MENTATaccuracy: o1 diagnosis 0.96 -> triage 0.46Hurt
vary patient gender: male vs female codedLMs · MENTAT monitoring/triage/documentationaccuracy: +10% monitoring, +8% triage, +10% documentation for male-codedMixed
describe patient as African American vs WhiteLMs · MENTAT diagnosisaccuracy: +5%Mixed
describe patient as Native American vs other ethnicitiesLMs · MENTAT treatmentaccuracy: +7 to 11%Mixed
use hierarchical Bradley-Terry model instead of regular for preference labels (ablation)annotation processing · MENTAT triage/documentationcreator answer in top-3: top-5 -> top-3Improved

Closest papers by meaning

Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.

Open scime draft

Get the next scime Dispatch and a monthly field update

Free. We send a confirmation link first; every email has a one-click unsubscribe. Privacy