Research map / Large language models / Language, safety and interpretability
JAG: A Crowdsourcing Framework for Joint Assessment and Peer Grading
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's title only (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| add open-response component (EM +open vs EM -open) (ablation) | JAG model · MTurk reading comprehension | answer correctness accuracy | Helped |
| add labeled answers for semi-supervised inference | EM +open · MTurk reading comprehension | accuracy | Helped |
| JAG model vs majority vote in low-competency classes | JAG model · synthetic data | accuracy: >10% | Helped |
| JAG model vs majority vote in high-competency classes | JAG model · synthetic data | accuracy | No clear effect |
| incorporate partially labeled answers for ranking students | EM +open · MTurk student ranking | Kendall-Tau | Helped |
| EM +open vs majority baseline for student ranking | EM +open · MTurk student ranking | Kendall-Tau | Mixed |
Closest papers by meaning
- Avoiding Imposters and Delinquents: Adversarial Crowdsourcing and Peer Prediction (NeurIPS 2016 · similarity 0.86)
- Pairwise HITS: Quality Estimation from Pairwise Comparisons in Creator-Evaluator Crowdsourcing Process (AAAI 2017 · similarity 0.84)
- Practical Sentiment Analysis for Education: The Power of Student Crowdsourcing (AAAI 2024 · similarity 0.82)
- Scaling Equitable Reflection Assessment in Education via Large Language Models and Role-Based Feedback Agents (AAAI 2026 · similarity 0.81)
- Unveiling the Tapestry of Automated Essay Scoring: A Comprehensive Investigation of Accuracy, Fairness, and Generalizability (AAAI 2024 · similarity 0.80)
- DataEnvGym: Data Generation Agents in Teacher Environments with Student Feedback (ICLR 2025 · similarity 0.80)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.