Research map / Large language models / Retrieval and knowledge
Knowledge Is Power: Harnessing Large Language Models for Enhanced Cognitive Diagnosis
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes KCD, a model-agnostic framework that uses ChatGPT (gpt-3.5-turbo-16k) to generate textual diagnoses of students and exercises and aligns them with CDM embeddings via contrastive learning and mask-reconstruction to fix cold-start weaknesses in cognitive diagnosis.
Problem
Cognitive diagnosis models perform poorly on infrequent (cold-start) students and exercises because they lack prior knowledge, and LLM semantic representations are not directly compatible with CDM behavioral representations.
Approach
Use an LLM to produce textual diagnoses of students and exercises from collaborative response logs, then align the text embeddings with CDM behavioral embeddings via InfoNCE contrastive learning (behavioral space alignment) or masked-reconstruction with dynamic masking (semantic space alignment).
Key results
- NCD-Beh improves NCD on Python from AUC 0.6522 to 0.6804 and ACC 0.7758 to 0.8007.
- KCD-Beh/Sem improve all seven CDMs on all four datasets, with the Beh variant generally best.
- Ablation on Python shows removing collaborative information, local or global contrast, or dynamic masking each lowers NCD-based performance.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Add behavioral space alignment to CDMs | CDMs in KCD · Python, Linux, Database, and Literature | AUC | Helped |
| Add semantic space alignment to CDMs | CDMs in KCD · Python, Linux, Database, and Literature | AUC | Helped |
| Add collaborative information during diagnosis generation (ablation) | NCD-Beh · Python | AUC: 0.6765 -> 0.6804 | Helped |
| Add collaborative information during diagnosis generation (ablation) | NCD-Sem · Python | AUC: 0.6637 -> 0.6687 | Helped |
| Add local contrast (ablation) | NCD-Beh · Python | AUC: 0.6726 -> 0.6804 | Helped |
| Add global contrast (ablation) | NCD-Beh · Python | AUC: 0.6748 -> 0.6804 | Helped |
| Use dynamic masking instead of a constant mask ratio (ablation) | NCD-Sem · Python | AUC: 0.6619 -> 0.6687 | Helped |
Surprising findings
- Behavioral space alignment outperforms semantic space alignment for most CDMs.
Limitations
- LLM input length limits prevent LLMs from fully processing detailed student-exercise response logs.
- Evaluated only on PTADisc courses with textual exercise content, and the experiments use a single LLM (gpt-3.5-turbo-16k).
Experimental setup
Datasets: PTADisc (Python, Linux, Database, Literature)
Benchmarks: PTADisc Python PTADisc Linux PTADisc Database PTADisc Literature
Metrics: AUC ACC RMSE
Baselines: IRT MIRT DINA NCD RCD SCD ACD
Builds on: NCD InfoNCE masked autoencoders ACD
Closest papers by meaning
- Multi-Perspective Consolidation Enhanced Cognitive Diagnosis via Conditional Diffusion Model (AAAI 2025 · similarity 0.92)
- Enhancing Cognitive Diagnosis Using Un-interacted Exercises: A Collaboration-Aware Mixed Sampling Approach (AAAI 2024 · similarity 0.90)
- Zero-1-to-3: Domain-Level Zero-Shot Cognitive Diagnosis via One Batch of Early-Bird Students towards Three Diagnostic Objectives (AAAI 2024 · similarity 0.88)
- Generating Computational Cognitive models using Large Language Models (NeurIPS 2025 · similarity 0.87)
- Collaborative Cognitive Diagnosis with Disentangled Representation Learning for Learner Modeling (NeurIPS 2024 · similarity 0.87)
- Boosting Neural Cognitive Diagnosis with Student’s Affective State Modeling (AAAI 2024 · similarity 0.87)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.