Research map / Reinforcement learning / Visual and representation RL

Personalized Exercise Recommendation with Semantically-Grounded Knowledge Tracing

NeurIPS 2025 · Reinforcement learning › Visual and representation RL · approach: RL with tree search for discovery · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes ExRec, an end-to-end pipeline (LLM KC annotation, contrastive question embeddings, calibrated LSTM knowledge tracing as RL environment) with a model-based value estimation critic for personalized exercise recommendation.

Problem

Existing RL-based exercise recommenders built on knowledge tracing ignore question semantics, use full exercise histories as states, require inference over all questions for rewards, and support only one RL algorithm.

Approach

Annotate questions with LLM-generated solution steps and KCs, learn semantic question/KC embeddings via contrastive learning, calibrate an LSTM KT model to predict KC-level knowledge states directly, and use the KT model's components to build a model-based value estimate for Q-learning continuous RL methods.

Key results

• MVE consistently boosts continuous value-based RL methods (DDPG, TD3, SAC) across all four tasks.
• On the weakest-KC task, MVE-enhanced continuous methods surpass discrete-action methods.
• Frozen semantic question embeddings reach AUC 81.26 vs 78.33 for ID-based initialization; KC calibration raises AUC to 81.65 and reduces KC-level MAE to 0.028.

Measured effects reported in the paper

Surprising findings

• Policy-based RL methods (TRPO, PPO) underperform value-based continuous methods across tasks.
• Existing manually annotated KCs in XES3G5M are often noisy, and LLM annotation yields more coherent KCs.

Limitations

• Relies on LLM-generated KC annotations and translated questions, whose quality was assessed mainly qualitatively.
• Evaluation is entirely in simulation using a KT model as the environment, not with real students.
• Only one KT architecture (LSTM) and one dataset (XES3G5M) are used.

Experimental setup

Datasets: XES3G5M

Benchmarks: XES3G5M (four exercise recommendation tasks)

Metrics: normalized knowledge improvement (% of max achievable) AUC micro-F1

Baselines: random policy historical data DDPG TD3 SAC TRPO PPO Discrete SAC C51 Rainbow DQN

Builds on: Tianshou pyKT DDPG SAC TD3

Closest papers by meaning

• GenAL: Generative Agent for Adaptive Learning (AAAI 2025 · similarity 0.86)
• Re2LLM: Reflective Reinforcement Large Language Model for Session-based Recommendation (AAAI 2025 · similarity 0.85)
• Predictive, scalable and interpretable knowledge tracing on structured domains (ICLR 2024 · similarity 0.85)
• Multi-type MOOCs Recommendation: Leveraging Deep Multi-Relational Representation and Hierarchical Reasoning (AAAI 2025 · similarity 0.84)
• ConvRec-R1: Training LLM-based Conversational Recommender Systems with Reinforcement Learning (ICLR 2026 · similarity 0.83)
• Enhancing Cognitive Diagnosis Using Un-interacted Exercises: A Collaboration-Aware Mixed Sampling Approach (AAAI 2024 · similarity 0.83)