Research map / Multi-agent systems and games / LLM agents
SketchMind: A Multi-Agent Cognitive Framework for Assessing Student-Drawn Scientific Sketches
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces SketchMind, a multi-agent framework that represents student-drawn scientific sketches as Bloom-annotated Sketch Reasoning Graphs (SRGs) to assess them and generate visual feedback, evaluated on 3,575 sketches across six NGSS-aligned items.
Problem
Automated assessment of free-form, visually diverse student-drawn scientific sketches is hard; existing approaches are image classifiers or monolithic vision-language models that lack interpretability, pedagogical alignment, and cognitive-level adaptability.
Approach
Model sketches as cognitively annotated semantic graphs (SRGs) with Bloom's taxonomy labels, and decompose assessment into four agents (rubric parsing, perception, cognitive alignment scoring, feedback and sketch modification) that compare student and gold-standard SRGs.
Key results
- GPT-4o accuracy rises from 55.6% without SRG to 77.1% with SRG (+21.4 average points).
- GPT-4.1 with SRG and multi-agent pipeline reaches 90.2% average accuracy, versus 77.4% without SRG.
- Human expert ratings average 4.1/5 for GPT-4.1 feedback and sketch modifications versus 2.3 for GPT-4o.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add SRG supervision | GPT-4o · Six science sketch items, average | accuracy: 55.6 -> 77.1 | Helped |
| Add SRG supervision | GPT-4.1 · Six science sketch items, average | accuracy: 77.4 -> 90.2 | Helped |
| Add SRG supervision | GPT-4.1-nano · Six science sketch items, average | accuracy: 62.0 -> 73.9 | Helped |
| Add SRG supervision | O3 · Six science sketch items, average | accuracy: 77.2 -> 89.0 | Helped |
| Add SRG supervision | O4-mini · Six science sketch items, average | accuracy: 73.3 -> 80.8 | Helped |
| Add SRG supervision | LLaMA 4 Scout · Six science sketch items, average | accuracy: 43.8 -> 64.8 | Helped |
| Add SRG supervision | LLaMA 4 Maverick · Six science sketch items, average | accuracy: 47.8 -> 72.4 | Helped |
| Replace single-agent reasoning with multi-agent reasoning without SRG (ablation) | GPT-4o · Six science sketch items, average | accuracy: 50.1 -> 55.6 | Helped |
| Replace single-agent reasoning with multi-agent reasoning with SRG (ablation) | GPT-4o · Six science sketch items, average | accuracy: 69.5 -> 77.1 | Helped |
| Replace single-agent reasoning with multi-agent reasoning without SRG (ablation) | GPT-4.1 · Six science sketch items, average | accuracy: 62.9 -> 77.4 | Helped |
| Replace single-agent reasoning with multi-agent reasoning with SRG (ablation) | GPT-4.1 · Six science sketch items, average | accuracy: 82.8 -> 90.2 | Helped |
| Use GPT-4.1 instead of GPT-4o for feedback and sketch modification | SketchMind · Six science sketch items | human feedback rating: 2.3 -> 4.1 | Helped |
Surprising findings
- Single-agent GPT-4.1 with SRG (82.8%) trails the multi-agent version (90.2%) by a wide margin, while o4-mini gains the least from SRG among the GPT models.
Limitations
- Inter-agent coordination is static and predefined rather than dynamically planned.
- Evaluation relies on overall sketch prediction accuracy, without SRG-level evaluation against expert-built graphs.
- Dataset release is pending approval; human evaluation covers a subset of outputs.
Experimental setup
Datasets: Zhai et al. NGSS-aligned student sketch dataset (3,575 sketches, six items)
Benchmarks: Zhai et al. NGSS-aligned student sketch dataset (six items R1-1, J2-1, M3-1, H4-1, H5-1, J6-1)
Metrics: sketch prediction accuracy human-rated feedback quality (1-5)
Baselines: GPT-4o GPT-4.1 GPT-4.1-nano o3 o4-mini LLaMA 4 Scout LLaMA 4 Maverick single-agent pipeline
Builds on: Betty's Brain NeRiF Bloom's taxonomy
Closest papers by meaning
- Visual Sketchpad: Sketching as a Visual Chain of Thought for Multimodal Language Models (NeurIPS 2024 · similarity 0.87)
- SketchThinker-R1: Towards Efficient Sketch-Style Reasoning in Large Multimodal Models (ICLR 2026 · similarity 0.87)
- From EduVisBench to EduVisAgent: A Benchmark and Multi-Agent Framework for Reasoning-Driven Pedagogical Visualization (ICLR 2026 · similarity 0.87)
- When Visualizing is the First Step to Reasoning: MIRA, a Benchmark for Visual Chain-of-Thought (CVPR 2026 · similarity 0.87)
- SketchAgent: Language-Driven Sequential Sketch Generation (CVPR 2025 · similarity 0.85)
- It's All About Your Sketch: Democratising Sketch Control in Diffusion Models (CVPR 2024 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.