Research map / Label-efficient and robust learning / Recognition and long-tail
From Crayons to Code: AI-Driven Insights into a Child’s Mental Health Through Drawings
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes an AI pipeline for screening children's Draw-A-Person drawings as 'need' or 'no need' referral, using GPT-4o prompt-based feature extraction with classical classifiers and a new Siamese++ network with an adaptive distance threshold to handle class imbalance.
Problem
Automating psychological screening of children from drawings (DAP test) under severe class imbalance (915 'no need' vs 37 'need' drawings).
Approach
Siamese++ replaces nearest-neighbor classification in a Siamese network with a cutoff distance Cα = αµ+ + (1−α)µ−, computed from mean negative-negative and positive-negative embedding distances, and also uses GPT-4o prompt engineering to extract expert-guided drawing features.
Key results
- Siamese++ reaches F1 0.87 (precision 0.85, recall 0.89) on the image dataset versus 0.78 for the nearest-neighbor Siamese and 0.00 for VGG16/ResNet transfer learning.
- With LLM-extracted GPT-4o features plus SMOTE, XGBoost achieves F1 0.85 and random forest F1 0.79; manual features gave XGBoost F1 0.76.
- Manhattan distance with 10 positive training samples gives F1 0.87, outperforming Euclidean (F1 0.795).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use LLM-extracted instead of manually extracted features with SMOTE | RF and XGB classifiers · DAP drawings | F1: RF: 0.76 -> 0.79; XGB: 0.76 -> 0.85 | Helped |
| Apply SMOTE to balance the classes | Traditional ML classifiers using drawing features · DAP drawings | F1: 0.00 -> 0.79; 0.00 -> 0.85 | Helped |
| Replace nearest-neighbor classification with an adaptive distance threshold | Siamese++ · DAP drawings, image classification | F1: 0.78 -> 0.87 | Helped |
| Use Manhattan instead of Euclidean distance (ablation) | Siamese++ · DAP drawings, 3, 5, and 10 positive training images | F1: 0.75 vs 0.62; 0.79 vs 0.70; 0.87 vs 0.795 | Helped |
| Increase positive training images from 3 to 10 (ablation) | Siamese++ with Manhattan distance · DAP drawings | F1: 0.75 -> 0.87 | Helped |
| Increase positive training images from 3 to 10 (ablation) | Siamese++ with Euclidean distance · DAP drawings | F1: 0.62 -> 0.795 | Helped |
| Set the adaptive-threshold parameter α around 0.5 (ablation) | Siamese++ · DAP drawings | F1 | Helped |
Surprising findings
- LLM-extracted features from GPT-4o were comparable to or better than manually extracted expert features for the classical classifiers.
- Transfer learning models (VGG16, ResNet) with augmentation produced zero precision, recall and F1 under class imbalance.
Limitations
- Very small positive class (37 'need' drawings) and a single dataset split, with results possibly sensitive to the split.
- Evaluation relies on a small test set; no confidence intervals or repeated runs are reported.
- Dataset is partly public Kaggle data and partly still being collected from local schools, with labels from a single psychology expert.
Experimental setup
Datasets: Draw-a-Person hand-drawn sketches (Kaggle) expert-labeled DAP drawings
Benchmarks: in-house DAP drawing dataset
Metrics: precision recall F1
Baselines: Random Forest SVM XGBoost VGG16 ResNet Siamese (nearest neighbor)
Builds on: Siamese network SMOTE transfer learning
Closest papers by meaning
- AACP: Aesthetics Assessment of Children’s Paintings Based on Self-Supervised Learning (AAAI 2024 · similarity 0.82)
- Unplugged K-12 AI Learning: Exploring Representation and Reasoning with a Facial Recognition Game (AAAI 2024 · similarity 0.82)
- Mental-Perceiver: Audio-Textual Multi-Modal Learning for Estimating Mental Disorders (AAAI 2025 · similarity 0.82)
- SketchMind: A Multi-Agent Cognitive Framework for Assessing Student-Drawn Scientific Sketches (NeurIPS 2025 · similarity 0.81)
- Self-Calibrating BCIs: Ranking and Recovery of Mental Targets Without Labels (NeurIPS 2025 · similarity 0.81)
- AI-generated Image Quality Assessment in Visual Communication (AAAI 2025 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.