Research map / Continual learning / Online and replay-based
Rethinking and Improving Student Learning and Forgetting Processes for Attention based Knowledge Tracing Models
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LefoKT, which adds relative forgetting attention (RPE-based temporal decay biases) to attention-based knowledge tracing models to separate forgetting from problem relevance and to extrapolate to longer interaction sequences.
Problem
Attention-based knowledge tracing models use uniform time-decay matrices that conflate forgetting with problem relevance, and fixed-length window training fails to model continuous forgetting over ever-growing student interaction sequences.
Approach
Replace time-decay matrices in attention with causal relative positional encoding biases (e.g., ALiBi, KERPLE, T5, FIRE, Sandwich) so that forgetting-related decay is decoupled from question relevance and depends only on relative distance, enabling length extrapolation.
Key results
- LefoKT-AKT-NF+ALiBi reaches 0.8317 AUC on AL2005 versus 0.8306 for AKT, with gains on BD2006 (0.8247 vs 0.8208) and NIPS34 (0.8045 vs 0.8033).
- LefoKT-RKT-NF with KERPLE-Power improves AUC over RKT by 0.96% on AL2005 (0.7857 vs 0.7761).
- In length-extrapolation tests trained on length 200, LefoKT maintains stable AUC up to length 1000 while AKT and AKT-NF degrade.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Replace AKT's forgetting module with ALiBi relative forgetting attention | AKT · AL2005 | AUC: 0.8306±0.0019 -> 0.8317±0.0021 | Helped |
| Replace AKT's forgetting module with ALiBi relative forgetting attention | AKT · BD2006 | AUC: 0.8208±0.0007 -> 0.8247±0.0006 | Helped |
| Replace AKT's forgetting module with ALiBi relative forgetting attention | AKT · NIPS34 | AUC: 0.8033±0.0003 -> 0.8045±0.0003 | Helped |
| Add KERPLE-Power while retaining RKT's forgetting module (ablation) | RKT · AL2005 | AUC: 0.7761±0.0010 -> 0.7837±0.0008 | Helped |
| Add KERPLE-Power while retaining RKT's forgetting module (ablation) | RKT · BD2006 | AUC: 0.7631±0.0013 -> 0.7735±0.0014 | Helped |
| Add KERPLE-Power while retaining RKT's forgetting module (ablation) | RKT · NIPS34 | AUC: 0.7966±0.0011 -> 0.8000±0.0002 | Helped |
| Add ALiBi while retaining AKT's forgetting module (ablation) | AKT · AL2005 | AUC: 0.8306±0.0019 -> 0.8310±0.0010 | No clear effect |
| Add ALiBi while retaining AKT's forgetting module (ablation) | AKT · BD2006 | AUC: 0.8208±0.0007 -> 0.8199±0.0008 | Hurt |
| Add ALiBi while retaining AKT's forgetting module (ablation) | AKT · NIPS34 | AUC: 0.8033±0.0003 -> 0.8032±0.0002 | No clear effect |
Surprising findings
- Adding RPEs to models that already have an exponential forgetting module decreased performance, which the authors interpret as RPEs capturing similar forgetting information.
- Attention-based models without an explicit forgetting module (RPE-only variants) outperformed the corresponding forgetting-module versions on several datasets.
Limitations
- Reported AUC gains over the best baseline are under 1% across datasets.
- Evaluated only on three KT datasets with fixed 200-length training for extrapolation tests.
Experimental setup
Datasets: AL2005 BD2006 NIPS34
Benchmarks: AL2005 BD2006 NIPS34
Metrics: AUC accuracy
Baselines: DKT DKT-Forget HawkesKT DKVMN LPKT DIMKT SAKT SAINT sparseKT DTransformer simpleKT RKT
Builds on: AKT RKT ALiBi KERPLE FIRE
Closest papers by meaning
- Cognitive Fluctuations Enhanced Attention Network for Knowledge Tracing (AAAI 2025 · similarity 0.89)
- Demystifying Language Model Forgetting with Low-rank Example Associations (NeurIPS 2025 · similarity 0.86)
- Uncertainty-aware Knowledge Tracing (AAAI 2025 · similarity 0.86)
- Spurious Forgetting in Continual Learning of Language Models (ICLR 2025 · similarity 0.86)
- Predicting the Susceptibility of Examples to Catastrophic Forgetting (ICML 2025 · similarity 0.85)
- Unlocking the Power of Function Vectors for Characterizing and Mitigating Catastrophic Forgetting in Continual Instruction Tuning (ICLR 2025 · similarity 0.85)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.