Research map / Large language models / Agents, code and math
Agent4Edu: Generating Learner Response Data by Generative Agents for Intelligent Education Systems
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Introduces Agent4Edu, an LLM-powered generative-agent simulator of learner response data with profile, memory (factual/short-/long-term with reflection and forgetting) and action modules, evaluated on a high-school math/physics dataset and CAT services.
Problem
Scarce offline practice response data and biased human online practice make it hard to simulate learner responses for evaluating and training personalized learning services (e.g., CAT), especially zero-shot.
Approach
LLM-based generative agents initialized from learner response data (practice-style statistics and IRT-estimated ability), with a psychology-inspired memory (reinforced factual memory, short-term and long-term memory, forgetting curve) and an action module that reads, identifies the knowledge concept, reasons through solutions, and predicts correctness, plus corrective reflection; agents interact w
Key results
- Agent4Edu (GPT-3.5-turbo) reaches 66.70% ACC and 79.84 F1 on EduData, the best among compared baselines (e.g., DAISIM 65.63 ACC, EERNN 65.72 ACC).
- Agent4Edu (GPT-4) on 100 learners reaches 82.43% ACC on exercise knowledge concept prediction versus 73.88% for GPT-3.5-turbo on all learners.
- Pretraining IRT with agent-generated data from zero-shot agents improves CAT F1-score by up to +2.39 (KLI, testing length 5 and 10) for FSI/KLI, while MAAT gains are marginal (+0.17 to +0.20).
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Train CAT IRT models with EduData+ augmented by Agent4Edu simulated responses | FSI · CAT, testing length 5 | F1-score: 80.11 -> 82.39; +2.28 | Helped |
| Train CAT IRT models with EduData+ augmented by Agent4Edu simulated responses | KLI · CAT, testing length 5 | F1-score: 79.45 -> 81.84; +2.39 | Helped |
| Train CAT IRT models with EduData+ augmented by Agent4Edu simulated responses | MAAT · CAT, testing length 5 | F1-score: 81.77 -> 81.97; +0.20 | Helped |
| Train CAT IRT models with EduData+ augmented by Agent4Edu simulated responses | FSI · CAT, testing length 10 | F1-score: 81.10 -> 82.51; +1.41 | Helped |
| Train CAT IRT models with EduData+ augmented by Agent4Edu simulated responses | KLI · CAT, testing length 10 | F1-score: 80.63 -> 82.82; +2.19 | Helped |
| Train CAT IRT models with EduData+ augmented by Agent4Edu simulated responses | MAAT · CAT, testing length 10 | F1-score: 81.71 -> 81.88; +0.17 | Helped |
| Remove profile, memory, memory enhancement, memory forgetting, or reflection (ablation) | Agent4Edu · Learner response prediction on EduData | ACC | Hurt |
| Remove profile, memory, memory enhancement, memory forgetting, or reflection (ablation) | Agent4Edu · Exercise-related knowledge prediction | ACC | No clear effect |
Surprising findings
- Agent4Edu with GPT-3.5-turbo achieves higher ACC than GPT-4 variants on the full comparison only when trained on all learners; GPT-4 on 100 learners scores lower than GPT-3.5 on full data for response prediction (66.51 vs 66.70 ACC) despite better knowledge prediction.
- Zero-shot agents' summaries were judged nearly indistinguishable from human-written ones, while exercise answers were less human-like.
Limitations
- GPT-4 experiments limited to 100 learners due to cost.
- Relies on a single proprietary dataset (EduData) from iFLYTEK, translated from Chinese to English.
- Zero-shot evaluation uses LLM-as-judge with GPT-3.5 annotators rather than human judges; the corrective reflection and tools are disabled in zero-shot.
Experimental setup
Datasets: EduData
Benchmarks: EduData
Metrics: ACC F1-score ROUGE-3 knowledge prediction ACC satisfaction AoD gain F1-score improvement
Baselines: KES DKVMN EERNN SAKT DAISIM FSI KLI MAAT
Builds on: Generative Agents IRT DNeuralCDM RCD chain-of-thought
Closest papers by meaning
- Developing LLM-Powered Trustworthy Agents for Personalized Learning Support (AAAI 2025 · similarity 0.89)
- GenAL: Generative Agent for Adaptive Learning (AAAI 2025 · similarity 0.87)
- DataEnvGym: Data Generation Agents in Teacher Environments with Student Feedback (ICLR 2025 · similarity 0.87)
- Scaling Equitable Reflection Assessment in Education via Large Language Models and Role-Based Feedback Agents (AAAI 2026 · similarity 0.85)
- MASTER: Enhancing Large Language Model via Multi-Agent Simulated Teaching (NeurIPS 2025 · similarity 0.85)
- Learn-by-interact: A Data-Centric Framework For Self-Adaptive Agents in Realistic Environments (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.