Research map / Architectures and efficiency / Attention and transformers
Cognitive Fluctuations Enhanced Attention Network for Knowledge Tracing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes FlucKT, an attention-based knowledge tracing model that separates long-term trends from short-term fluctuations with causal-convolution decomposition and penalizes long-range attention with a kernelized bias to improve length generalization.
Problem
Attention-based knowledge tracing models over-smooth short-term cognitive fluctuations and degrade when test sequences are longer than training sequences (train short, test long).
Approach
A decomposition layer (causal convolution splits embeddings into long-term trend and short-term fluctuation components, reweighted with learnable parameters) plus a learnable kernelized distance bias (log-based penalty on |i-j|) added to attention scores to focus attention locally and generalize to longer sequences.
Key results
- On AL2005, FlucKT reaches AUC 0.8376 at window 200 and 0.8358 at window 1000, above the best baseline extraKT (0.8317).
- On BD2006, FlucKT achieves AUC 0.8269 at window 200 and 0.8250 at window 1000, versus 0.8247 for extraKT at window 200.
- On NIPS34, FlucKT achieves AUC about 0.8047 across all window sizes, matching or slightly exceeding extraKT; the improvements on the other datasets are under 1% AUC.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use FlucKT instead of extraKT | Knowledge tracing · AL2005 | AUC: 0.8317 -> 0.8376 | Helped |
| Use FlucKT instead of extraKT | Knowledge tracing · BD2006 | AUC: 0.8247 -> 0.8269 | Helped |
| Use FlucKT instead of extraKT | Knowledge tracing · NIPS34 | AUC: 0.8045 -> 0.8047 | Helped |
| Add the Cognitive Feature Decomposition layer (ablation) | FlucKT · AL2005 | AUC | Helped |
| Add the Cognitive Feature Decomposition layer (ablation) | FlucKT · BD2006 | AUC | Helped |
| Add the Cognitive Feature Decomposition layer (ablation) | FlucKT · NIPS34 | AUC | Helped |
| Add kernelized bias-enhanced attention (ablation) | FlucKT · AL2005 | AUC | Helped |
| Add kernelized bias-enhanced attention (ablation) | FlucKT · BD2006 | AUC | Helped |
| Add kernelized bias-enhanced attention (ablation) | FlucKT · NIPS34 | AUC | Helped |
| Add the Cognitive Feature Decomposition layer (ablation) | AKT · Three real-world datasets | AUC | Helped |
| Add the Cognitive Feature Decomposition layer (ablation) | SimpleKT · Three real-world datasets | AUC | Helped |
Surprising findings
- Existing attention-based KT models such as SAKT, SAINT and SimpleKT drop sharply in AUC when test sequence length exceeds the training length of 200.
- Adding the decomposition layer also improves AKT and SimpleKT, notably SimpleKT's length generalization over 200-1000 steps.
Limitations
- AUC gains over the strongest baseline are small (under 1%) on the reported datasets.
- On NIPS34, which has the fewest interactions per student, gains are less prominent.
Experimental setup
Datasets: AL2005 BD2006 NIPS34
Benchmarks: AL2005 BD2006 NIPS34
Metrics: AUC ACC
Baselines: DKT DKVMN GKT SAKT SAINT AKT ATKT LPKT SimpleKT AT-DKT FoLiBiKT DTransformer
Builds on: AKT SimpleKT ALiBi (FoLiBi/linear bias) pyKT
Closest papers by meaning
- Rethinking and Improving Student Learning and Forgetting Processes for Attention based Knowledge Tracing Models (AAAI 2025 · similarity 0.89)
- Uncertainty-aware Knowledge Tracing (AAAI 2025 · similarity 0.85)
- Extended LSTMs for Knowledge Tracing: Peeking Inside the Black Box (Student Abstract) (AAAI 2025 · similarity 0.84)
- Zoology: Measuring and Improving Recall in Efficient Language Models (ICLR 2024 · similarity 0.84)
- Tracing the Traces: Latent Temporal Signals for Efficient and Accurate Reasoning (ICLR 2026 · similarity 0.84)
- Enhancing Cognitive Diagnosis Using Un-interacted Exercises: A Collaboration-Aware Mixed Sampling Approach (AAAI 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.