Research map / Large language models / AI and society
From Consumers to Critical Users: Prompty, an AI Literacy Tool for High School Students
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Prompty, a co-designed web-based AI literacy tool built on text-davinci-003 that has high school students compare three LLM outputs, scaffold prompts, and use limited tries, evaluated through a single-teacher pilot debrief.
Problem
Students lack skills to critically evaluate and effectively create text with LLMs, and teachers need classroom-ready AI literacy tools that fit their disciplinary goals.
Approach
A classroom LLM writing tool that combines contrasting cases (three simultaneous outputs), Role/Context/Task prompt guides as computer-based scaffolds, limited generation tries to encourage intentional use, and moderation guardrails, co-designed with an ELA teacher.
Key results
- Pilot in two ELA high school classes (ages 14-16) over a 3-week writing project; evidence is the teacher's qualitative Zoom debrief, not student outcome measures.
- Teacher reported students quickly understood the interface and that contrasting three outputs drove compare-and-evaluate objectives.
- Teacher noted some students struggled to revise prompts and several requested more tries.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Present three LLM responses side by side | Prompty · High school English class | student comparison and evaluation | Helped |
| Limit the number of generation tries | Prompty · High school English class | intentionality and reflection | Helped |
| Scaffold prompt writing with Role, Context, and Task guides | Prompty · High school English class | prompt-writing process | Helped |
| Provide more guidance for revising prompts based on weaknesses in generated results | Prompty · High school English class | meaningful prompt iteration | Mixed |
Surprising findings
- Students with the best responses wrote paragraph-length prompts, shifting the teacher's view of prompting as a disciplinary writing skill.
Limitations
- Evidence comes solely from one teacher's debrief; researchers did not observe classes in person.
- Relies on a deprecated model (text-davinci-003) and a simple interface not suited to chat.
- No student-level learning outcome measures reported.
Experimental setup
Builds on: Contrasting Cases distributed cognition OpenAI Moderation endpoint
Closest papers by meaning
- Word2Vec4Kids: Interactive Challenges to Introduce Middle School Students to Word Embeddings (AAAI 2025 · similarity 0.87)
- An Effectiveness Study of Teacher-Led AI Literacy Curriculum in K-12 Classrooms (AAAI 2024 · similarity 0.87)
- “From Unseen Needs to Classroom Solutions”: Exploring AI Literacy Challenges & Opportunities with Project-Based Learning Toolkit in K-12 Education (AAAI 2025 · similarity 0.86)
- Co-designing AI Education Curriculum with Cross-Disciplinary High School Teachers (AAAI 2024 · similarity 0.86)
- Supporting AI Literacy Teaching Through the Development of Assessments for Classroom Use (AAAI 2025 · similarity 0.86)
- Understanding K-12 Teachers’ Needs for AI Education: A Survey-Based Study (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.