Research map / Label-efficient and robust learning / Recognition and long-tail
Explicit and Implicit Examinee-Question Relation Exploiting for Efficient Computerized Adaptive Testing
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Proposes RECAT, a generative question selector that uses implicit examinee-question relations (via a CD-specific generator trained with policy-gradient and contrastive losses) to produce a candidate set for policy-based CAT selectors, improving accuracy and cutting selection latency.
Problem
Computerized adaptive testing must choose informative questions; policy-based selectors use explicit examinee-question relations but are slow over large banks and greedy, while learning-based selectors are fast but adapt poorly to newly added questions.
Approach
Define a true-ability-oriented best-fitting question objective; train K question generators conditioned on estimated ability to produce best-fitting question parameters, match them to real questions via KL divergence to form a small candidate set, and let existing policy-based selectors (MAAT, BECAT, Random) pick from that set.
Key results
- RECAT-MAAT and RECAT-BECAT outperform baselines in AUC/ACC on all four datasets at steps 5 and 10 (e.g., Math step 10 AUC/ACC 0.7562/0.6945 for RECAT-MAAT vs 0.7524/0.6933 for GMOCAT best baseline).
- On NIPS-EDU at step 10, selection latency drops from 239.63s (BECAT) to 8.41s (RECAT-BECAT); on ASSIST12 at step 10 from 1432.15s to 26.15s.
- RECAT-based approaches consistently achieve the lowest MSE in ability estimation simulation on ASSIST12 and NIPS-EDU, while original MAAT shows unstable error decreases.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Combine implicit question-relation learning with MAAT selection | RECAT-MAAT · ASSIST09, step 5 | AUC: 0.6443 -> 0.6485 | Helped |
| Combine implicit question-relation learning with BECAT selection | RECAT-BECAT · NIPS-EDU, step 10 | AUC: 0.7314 -> 0.7423 | Helped |
| Use implicit relation to narrow MAAT's question search | RECAT-MAAT · ASSIST12, step 5 | selection latency: 49.04/0.49 | Helped |
| Use implicit relation to narrow BECAT's question search | RECAT-BECAT · NIPS-EDU, step 10 | selection latency: 239.63/8.41 | Helped |
| Add more questions absent during training to the question bank | RECAT-MAAT · NIPS-EDU new-question evaluation | selection performance | Hurt |
Surprising findings
- Original MAAT shows negative optimization of examinee ability from step 5 to step 10 on NIPS-EDU, attributed to greedy local optima.
- Removing questions from the bank during training degrades RECAT-MAAT performance, since removed questions may have carried valuable diagnostic signal for the knowledge-matching process.
Limitations
- Adaptability to new questions degrades as more questions are added without retraining, per RQ4 experiment.
- True ability is approximated by diagnosis over the full bank on simulated response logs.
Experimental setup
Datasets: ASSIST09 ASSIST12 NIPS-EDU Math
Benchmarks: ASSIST09 ASSIST12 NIPS-EDU Math
Metrics: AUC ACC MSE selection latency
Baselines: Random MAAT BECAT BOBCAT NCAT GMOCAT
Builds on: MAAT BECAT NCAT BOBCAT NCD
Closest papers by meaning
- Computerized Adaptive Testing via Collaborative Ranking (NeurIPS 2024 · similarity 0.86)
- A Unified Adaptive Testing System Enabled by Hierarchical Structure Search (ICML 2024 · similarity 0.85)
- EduDiag: A Benchmark for Educational Diagnostic Reasoning with Error Tracing and Correction on Large Multimodal Models (CVPR 2026 · similarity 0.82)
- A Semantic-Sampling Framework for Evaluating Calibration in Open-Ended Question Answering (NeurIPS 2026 · similarity 0.82)
- Thinker: Learning to Think Fast and Slow (NeurIPS 2025 · similarity 0.82)
- QFFT, Question-Free Fine-Tuning for Adaptive Reasoning (NeurIPS 2025 · similarity 0.82)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.