Research map / Large language models / AI and society
Advancing Research on Equitable AI Education Through a Focus on Implementation: Insights from a Middle School Computer Vision Module Beta-Test
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Reports a design-based implementation research study of the RAICA computer vision module beta-tested by 12 educators and about 282 middle/high school students across nine sites in four countries, analyzing fidelity-of-implementation data from in-document teacher comments.
Problem
Implementation data for AI education curricula is scarce, so it is unclear which curricular components are implemented as designed, how implementation varies across diverse contexts, and how teachers respond to curriculum components.
Approach
Embeds a custom Fidelity Tracker into a cloud-based teacher guide as multiple-choice and open-response comments, then analyzes fidelity, adaptations, and teacher feedback by component, concept, and context within a DBIR framework.
Key results
- Five of nine sites completed all eight parts; activities were implemented 'as prescribed' 40.7% of the time, assessments 37.5%.
- Teachers responded to multiple-choice questions 77.78% of the time and produced 225 qualitative responses across the module.
- Teacher notes were rated as supportive 73.9% of the time; U.S. sites skipped activities 17% of the time vs 3% at international sites.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Implement the module at U.S. sites rather than international sites | RAICA · Computer vision module, activity implementation | activities skipped: 17 vs 3% of the time | Mixed |
| Implement the module at U.S. sites rather than international sites | RAICA · Computer vision module, assessment implementation | assessment fidelity: 27% vs. 50% | Hurt |
| Adapt activities to local classroom needs | RAICA activities · Computer vision module | learning objective impact: nine out of 42 positive; six ... negative | Mixed |
| Modify assessment content or format | RAICA assessments · Computer vision module | learning objective impact: three ... positive, and two ‘negative’ | Mixed |
| Use teacher notes as instructional support | RAICA teachers · Computer vision module | teacher-rated support: 73.9% | Helped |
Surprising findings
- Activities 4.5.0 (acting out a neural network) and 8.3.0 (project showcase) were never implemented as prescribed by any site.
- Responsible design components were modified 75% of the time and never implemented as intended.
Limitations
- Fidelity data is self-reported and subject to bias and inconsistency.
- Maximal variation sampling limits analysis of setting types beyond U.S. vs international.
- Inter-coder reliability was not established.
Experimental setup
Datasets: Fidelity Tracker responses from RAICA computer vision module beta-test
Metrics: fidelity completion rate proportion of prescribed vs modified components teacher comment counts and codes teacher note agreement rate
Builds on: Design-Based Implementation Research TPACK Universal Design for Learning
Closest papers by meaning
- Supporting AI Literacy Teaching Through the Development of Assessments for Classroom Use (AAAI 2025 · similarity 0.90)
- “From Unseen Needs to Classroom Solutions”: Exploring AI Literacy Challenges & Opportunities with Project-Based Learning Toolkit in K-12 Education (AAAI 2025 · similarity 0.90)
- Empowering Educators in AI: Insights from Co-Designing an AI Microcredential with and for K-12 Educators (AAAI 2025 · similarity 0.89)
- Understanding K-12 Teachers’ Needs for AI Education: A Survey-Based Study (AAAI 2025 · similarity 0.89)
- An Effectiveness Study of Teacher-Led AI Literacy Curriculum in K-12 Classrooms (AAAI 2024 · similarity 0.89)
- ImageSTEAM: Teacher Professional Development for Integrating Visual Computing into Middle School Lessons (AAAI 2024 · similarity 0.88)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.