Research map / Large language models / Language, safety and interpretability
EduLLM: Leveraging Large Language Models and Framelet-Based Signed Hypergraph Neural Networks for Student Performance Prediction
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes EduLLM, combining LLM-derived semantic embeddings with a framelet-based signed hypergraph neural network (FraS-HNN) to predict student performance on MCQs.
Problem
Predicting students' future performance on multiple-choice questions while capturing higher-order student-question interactions and the semantics of question content, which pairwise GNNs and temporal models miss.
Approach
Formulates student-MCQ responses as a signed hypergraph (positive hyperedges for correct answers, negative for incorrect) and applies Haar-type framelet low-pass and high-pass spectral filters with Chebyshev approximation, plus LLM-derived question semantic embeddings.
Key results
- EduLLM reaches binary-F1 of 0.945 on Law and 0.829 on Sydney23146, outperforming the best baseline LLM-SBCL (0.908 and 0.760).
- Replacing FraS-HNN with other hypergraph networks (HGNN, HyperGCN, AllSetTransformer, ED-HNN, SheafHyperGNN) lowers binary-F1 on all five datasets.
- Ablations show removing high-pass, low-pass, or LLM embeddings each reduces binary-F1 on every dataset.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add the high-pass filter to capture fine-grained distinctions (ablation) | EduLLM · Biology, Law, Cardiff, Sydney19, Sydney23 | Binary-F1: 0.801 ± 0.005 -> 0.809 ± 0.010; 0.943 ± 0.017 -> 0.945 ± 0.005; 0.731 ± 0.019 -> 0.753 ± 0.011; 0.705 ± 0.014 -> 0.712 ± | Helped |
| add the low-pass filter to model shared preferences and global patterns (ablation) | EduLLM · Biology, Law, Cardiff, Sydney19, Sydney23 | Binary-F1: 0.799 ± 0.005 -> 0.809 ± 0.010; 0.934 ± 0.029 -> 0.945 ± 0.005; 0.741 ± 0.017 -> 0.753 ± 0.011; 0.693 ± 0.006 -> 0.712 ± | Helped |
| add LLM-derived semantic embeddings (ablation) | EduLLM · Biology, Law, Cardiff, Sydney19, Sydney23 | Binary-F1: 0.795 ± 0.017 -> 0.809 ± 0.010; 0.941 ± 0.013 -> 0.945 ± 0.005; 0.740 ± 0.004 -> 0.753 ± 0.011; 0.703 ± 0.033 -> 0.712 ± | Helped |
| use FraS-HNN instead of SheafHyperGNN (ablation) | EduLLM · Sydney19351, Sydney23146, Biology, Cardiff20102, Law | Binary-F1: 0.684 ± 0.023 -> 0.712 ± 0.016; 0.711 ± 0.030 -> 0.829 ± 0.006; 0.732 ± 0.022 -> 0.809 ± 0.010; 0.687 ± 0.025 -> 0.753 ± | Helped |
| use EduLLM instead of LLM-SBCL | student performance prediction · Biology, Law, Cardiff20102, Sydney19351, Sydney23146 | Binary-F1: 0.787 ± 0.014 -> 0.809 ± 0.010; 0.908 ± 0.018 -> 0.945 ± 0.005; 0.734 ± 0.023 -> 0.753 ± 0.011; 0.694 ± 0.021 -> 0.712 ± | Helped |
Surprising findings
- The LLM embeddings were not computed by the authors; pre-encoded semantic embeddings from prior work were used, so different LLM backbones could not be compared directly.
- Robustness to LLM choice was approximated by adding Gaussian noise to the pre-encoded embeddings rather than swapping LLMs.
Limitations
- LLM semantic embeddings were pre-encoded by prior work, so the LLM component was not directly evaluated.
- Relationship to knowledge tracing and cognitive diagnosis settings is left for future work.
- Robustness analysis uses synthetic noise as a proxy for different LLMs.
Experimental setup
Datasets: Biology (University of Auckland) Law (University of Auckland) Cardiff20102 (Cardiff University) Sydney19351 (University of Sydney) Sydney23146 (University of Sydney)
Benchmarks: Biology Law Cardiff20102 Sydney19351 Sydney23146
Metrics: AUC binary-F1
Baselines: Random GCN GAT SGCN SBGNN SBCL LLM-SBCL HGNN HyperGCN AllDeepSets AllSetTransformer ED-HNN
Builds on: framelet theory signed bipartite graph contrastive learning initial residual and identity mapping (GCNII) HGNN
Closest papers by meaning
- Enhancing Student Performance Prediction on Learnersourced Questions with SGNN-LLM Synergy (AAAI 2024 · similarity 0.90)
- Toward Student-oriented Teacher Network Training for Knowledge Distillation (ICLR 2024 · similarity 0.85)
- An Autoencoder-Like Nonnegative Matrix Co-Factorization for Improved Student Cognitive Modeling (NeurIPS 2024 · similarity 0.84)
- K12-KGraph: A Curriculum-Aligned Knowledge Graph for Benchmarking and Training Educational LLMs (NeurIPS 2026 · similarity 0.84)
- MiniPLM: Knowledge Distillation for Pre-training Language Models (ICLR 2025 · similarity 0.83)
- DistiLLM-2: A Contrastive Approach Boosts the Distillation of LLMs (ICML 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.