Research map / Large language models / AI and society
Assessing the Quality of AI-Generated Exams: A Large-Scale Field Study
Calvin Isley, Joshua Gilbert, Evangelos Kassos, Michaela Kocher, Allen Nie, Emma Brunskill, Ben Domingue, Jake Hofman, Joscha Legewie, Teddy Svoronos, Charlotte Tuminelli, Sharad Goel
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| use AI-generated questions instead of AP Statistics questions | AI-generated exams · item difficulty | mean difficulty beta: 0.35 -> -0.45 | Mixed |
| use AI-generated questions instead of AP Statistics questions | AI-generated exams · item discrimination | mean discrimination alpha: 1.2 -> 1.3 | No clear effect |
| use AI-generated exams instead of AP Statistics exams | AI-generated exams · test information | maximum test information: 2.61 -> 3.85 | Improved |
| use AI-generated exams instead of AP Statistics exams | AI-generated exams · test reliability | reliability at peak: 0.72 -> 0.79 | Improved |
| use AI-generated exams instead of AP Statistics exams | AI-generated exams · test information at theta=0 | test information: 2.5 -> 3.36 | Improved |
| use AI-generated questions instead of AP Statistics questions | AI-generated exams · share of highly or very highly discriminating items | share of items (%): 21 -> 36 | Improved |
Closest papers by meaning
- LLMs in the Classroom: Outcomes and Perceptions of Questions Written with the Aid of AI (AAAI 2025 · similarity 0.90)
- How Teachers Can Use Large Language Models and Bloom’s Taxonomy to Create Educational Quizzes (AAAI 2024 · similarity 0.90)
- Students’ Perceptions and Preferences of Generative Artificial Intelligence Feedback for Programming (AAAI 2024 · similarity 0.87)
- Automated Evaluation of Retrieval-Augmented Language Models with Task-Specific Exam Generation (ICML 2024 · similarity 0.87)
- DataEnvGym: Data Generation Agents in Teacher Environments with Student Feedback (ICLR 2025 · similarity 0.86)
- Automated Assessment of Student Self-Explanation in Code Comprehension Using Pre-Trained Language Models (AAAI 2025 · similarity 0.86)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
