Research map / Large language models / Retrieval and knowledge
Enhancing Student Performance Prediction on Learnersourced Questions with SGNN-LLM Synergy
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes LLM-SBCL, which combines signed bipartite graph contrastive learning over student-MCQ answer links with GPT-3.5-extracted knowledge-point embeddings to predict student performance on learnersourced MCQs, improving cold-start prediction.
Problem
Predicting student correctness on learnersourced multiple-choice questions is hard due to noisy student-generated data and cold-start conditions where new questions have few responses.
Approach
Models student-question answer correctness as edge sign prediction on a signed bipartite graph with contrastive learning over graph augmentations, and adds LLM-extracted knowledge-point embeddings of question text to address cold start.
Key results
- On five PeerWise datasets, LLM-SBCL reaches binary-F1 0.787 (Biology), 0.908 (Law), 0.734 (Cardiff), 0.694 (Sydney1), 0.76 (Sydney2), outperforming SBCL and other baselines.
- In the cold-start setting, LLM-SBCL improves binary-F1 by about 6.4% on average over SBCL.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add LLM-derived semantic question embeddings to SBCL | SBCL · Biology | binary-F1: 0.772±0.016 -> 0.787±0.014 | Helped |
| Add LLM-derived semantic question embeddings to SBCL | SBCL · Law | binary-F1: 0.901±0.016 -> 0.908±0.018 | Helped |
| Add LLM-derived semantic question embeddings to SBCL | SBCL · Cardiff20102 | binary-F1: 0.718±0.018 -> 0.734±0.023 | Helped |
| Add LLM-derived semantic question embeddings to SBCL | SBCL · Sydney19351 | binary-F1: 0.674±0.021 -> 0.694±0.021 | Helped |
| Add LLM-derived semantic question embeddings to SBCL | SBCL · Sydney23146 | binary-F1: 0.733±0.019 -> 0.76±0.022 | Helped |
| Add LLM-derived semantic question embeddings to SBCL | SBCL · Biology cold start | binary-F1: 0.616±0.049 -> 0.629±0.061 | Helped |
| Add LLM-derived semantic question embeddings to SBCL | SBCL · Law cold start | binary-F1: 0.885±0.039 -> 0.894±0.031 | Helped |
| Add LLM-derived semantic question embeddings to SBCL | SBCL · Cardiff cold start | binary-F1: 0.595±0.08 -> 0.625±0.045 | Helped |
| Add LLM-derived semantic question embeddings to SBCL | SBCL · Sydney1 cold start | binary-F1: 0.573±0.057 -> 0.623±0.055 | Helped |
| Add LLM-derived semantic question embeddings to SBCL | SBCL · Sydney2 cold start | binary-F1: 0.779±0.039 -> 0.801±0.009 | Helped |
| Add LLM-derived semantic question embeddings to SBCL | SBCL · Law cold start | macro-F1: 0.559±0.025 -> 0.548±0.017 | Hurt |
| Add LLM-derived semantic question embeddings to SBCL | SBCL · Sydney2 cold start | AUC: 0.518±0.017 -> 0.511±0.013 | Hurt |
Surprising findings
- On Law and Sydney2, SBCL without LLM embeddings has higher macro-F1 and AUC than LLM-SBCL, so the LLM gain is not uniform across metrics.
Limitations
- Cold-start evaluation simulates new questions by holding out 10% of questions, not truly new items.
- Relies on a proprietary LLM (gpt-3.5-turbo-0613) for knowledge-point extraction, with no analysis of prompt sensitivity or cost.
- Mixed results on macro-F1 and AUC for some datasets.
Experimental setup
Datasets: PeerWise biology PeerWise law PeerWise cardiff20102 PeerWise sydney19351 PeerWise sydney23146
Benchmarks: PeerWise biology PeerWise law PeerWise cardiff20102 PeerWise sydney19351 PeerWise sydney23146
Metrics: binary-F1 micro-F1 macro-F1 AUC
Baselines: Random Embedding GCN GAT SGCN SBGNN SBCL
Builds on: SBCL SGCN SBGNN GAT InfoNCE
Closest papers by meaning
- EduLLM: Leveraging Large Language Models and Framelet-Based Signed Hypergraph Neural Networks for Student Performance Prediction (ICML 2025 · similarity 0.90)
- Beyond the Golden Teacher: Enhancing Graph Learning through LLM-GNN Co-teaching (NeurIPS 2026 · similarity 0.84)
- Exploring Iterative Enhancement for Improving Learnersourced Multiple-Choice Question Explanations with Large Language Models (AAAI 2025 · similarity 0.84)
- Question Calibration and Multi-Hop Modeling for Temporal Question Answering (AAAI 2024 · similarity 0.84)
- Graph Neural Prompting with Large Language Models (AAAI 2024 · similarity 0.84)
- Toward Student-oriented Teacher Network Training for Knowledge Distillation (ICLR 2024 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.