Research map / Large language models / Agents, code and math
A Dataset for Learning University STEM Courses at Scale and Generating Questions at a Human Level
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use the specialized astronomy-trained model instead of the generalized Codex approach | Question solving · Introduction to Astronomy | accuracy: 67% -> 92% | Helped |
| Add chain-of-thought prompting | GPT-3 (text-davinci-003) · Introduction to Astronomy | accuracy: 30% -> 37.5% | Helped |
| Add chain-of-thought prompting | GPT-3 (text-davinci-002) · Introduction to Astronomy | accuracy: 15% -> 27% | Helped |
| Use a model trained on machine-learning problems for astronomy questions | Specialized ML model · Introduction to Astronomy | accuracy: 0% | Hurt |
| Use Wolfram Alpha to solve the astronomy problems | Wolfram Alpha · Introduction to Astronomy | accuracy: 0% | Hurt |
| Generate questions with the generalized approach | University STEM course questions · Student survey: machine-generated questions | rated appropriate for course: 86.1% -> 89.2% | Helped |
| Generate questions with the generalized approach | University STEM course questions · Student survey: machine-generated versus human-written questions | difficulty rating: 2.94 with a 0.90 standard deviation -> 2.75 with a 0.89 standard deviation | Mixed |
| Generate questions with the generalized approach | University STEM course questions · Student survey: machine-generated questions | identified correctly as machine-generated: 37.0% | Hurt |
Closest papers by meaning
- NaturalReasoning: Reasoning in the Wild with 2.8M Challenging Questions (NeurIPS 2025 · similarity 0.83)
- Measuring Vision-Language STEM Skills of Neural Models (ICLR 2024 · similarity 0.82)
- How Teachers Can Use Large Language Models and Bloom’s Taxonomy to Create Educational Quizzes (AAAI 2024 · similarity 0.82)
- Language Models as Science Tutors (ICML 2024 · similarity 0.82)
- Learning to Solve Complex Problems via Dataset Decomposition (NeurIPS 2025 · similarity 0.82)
- MathScale: Scaling Instruction Tuning for Mathematical Reasoning (ICML 2024 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.