Research map / Large language models / Retrieval and knowledge
Developing LLM-Powered Trustworthy Agents for Personalized Learning Support
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes MerryQuery, an LLM-powered educational agent combining RAG, rule-based content control, and RLHF with a learning profile and multi-step verification, reported as a prototype piloted in one classroom with no quantitative results.
Problem
LLM educational tools suffer from hallucinations, weak contextual relevance, and limited personalization, hindering trustworthy classroom adoption.
Approach
Combine retrieval-augmented generation grounded in course materials and student data with instructor-defined pre-retrieval and post-generation rules, a dynamic learning profile, a memory mechanism, and RLHF from student and teacher feedback, plus a guided-learning dialogue that scaffolds rather than gives answers.
Key results
- Preliminary pilot in the CSC113 MATLAB course reported improved response reliability and student understanding, without specific numbers.
- Prompt engineering enforced teacher-defined answer-depth rules and prevented disclosure of complete solutions in the prototype.
- A controlled experiment against ChatGPT and an ablation study were planned but not yet reported.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use prompt engineering to enforce teacher-defined answer-depth rules | MerryQuery · CSC113 - Introduction to Computing (MATLAB) | compliance with teacher policies | Helped |
| Use RAG to ground responses in course materials and verified data | MerryQuery · CSC113 pilot and internal testing | accuracy on complex PDFs | Helped |
| Add a memory mechanism for coherent conversations | MerryQuery · CSC113 pilot and internal testing | memory performance | Helped |
Surprising findings
- Prompt engineering alone prevented disclosure of complete solutions, without needing the separate post-generation verification agent.
Limitations
- Evaluation is preliminary; controlled experiment and ablation study are planned but not completed.
- Pilot limited to a single course and a usability study.
- Certain areas of the prototype still need refinement according to the author.
Experimental setup
Datasets: course materials (e.g., syllabus) student profiles and historical data course forum data with verified answers (planned)
Metrics: student engagement accuracy perceived trustworthiness response reliability student understanding
Baselines: ChatGPT
Builds on: Retrieval-Augmented Generation RLHF Direct Preference Optimization
Closest papers by meaning
- MerryQuery: A Trustworthy LLM-Powered Tool Providing Personalized Support for Educators and Students (AAAI 2025 · similarity 0.92)
- Agent4Edu: Generating Learner Response Data by Generative Agents for Intelligent Education Systems (AAAI 2025 · similarity 0.89)
- GenAL: Generative Agent for Adaptive Learning (AAAI 2025 · similarity 0.86)
- Enhancing Personalized Multi-Turn Dialogue with Curiosity Reward (NeurIPS 2025 · similarity 0.86)
- RLPF: Reinforcement Learning from Prediction Feedback for User Summarization with LLMs (AAAI 2025 · similarity 0.85)
- How to Train Your LLM Web Agent: A Statistical Diagnosis (NeurIPS 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.