Research map / Label-efficient and robust learning / Zero-/few-shot and meta-learning
Zero Shot Learning for Code Education: Rubric Sampling with Deep Learning Inference
Mike Wu, Milan Mosse, Noah Goodman, Chris Piech
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use rubric sampling to train with synthetic labeled programs | Code feedback prediction · P1 body | F1: +0.31 | Improved |
| Use rubric sampling to train with synthetic labeled programs | Code feedback prediction · P1 tail | F1: +0.13 | Improved |
| Combine rubric sampling with an MVAE | Rubric sampling code feedback model · P1 and P8 | F1: an additional 0.2 | Improved |
| Use a novice-authored rubric instead of an expert-authored rubric (ablation) | Rubric sampling · P1 | F1: 0.69 ± 0.03 -> 0.63 ± 0.01 | Hurt |
| Use hand-picked PCFG probabilities instead of random probabilities (ablation) | Rubric sampling · P1 | F1: 0.26 ± 0.002 increase | Improved |
| Learn PCFG probabilities with evolutionary strategies from random initialization (ablation) | Rubric sampling · P1 | F1 difference from hand-picked probabilities: 0.007 ± 0.005 | Improved |
| Learn PCFG probabilities with evolutionary strategies (ablation) | Rubric sampling · P8 | F1 relative to expert parameters: 0.005 ± 0.009 | Improved |
| Train the MVAE using unlabeled student programs as well as synthetic labeled programs (ablation) | Rubric sampling with MVAE · Code.org student-program distribution | distribution match | Improved |
| Use raw frequency counts instead of the log-Zipf transformation (ablation) | MVAE program generation · Code.org student programs | coverage of program frequency distribution | Hurt |
| Use hand-designed PCFG synthetic data alone instead of the MVAE trained with unlabeled data (ablation) | Synthetic student-program generation · Code.org student-program distribution | distribution match | Hurt |
Closest papers by meaning
- Hints-In-Browser: Benchmarking Language Models for Programming Feedback Generation (NeurIPS 2024 · similarity 0.84)
- Automated Assessment of Student Self-Explanation in Code Comprehension Using Pre-Trained Language Models (AAAI 2025 · similarity 0.83)
- Primal Generation, Dual Judgment: Self-Training from Test-Time Scaling (NeurIPS 2026 · similarity 0.83)
- CurBench: Curriculum Learning Benchmark (ICML 2024 · similarity 0.83)
- Benchmarking Generative Models on Computational Thinking Tests in Elementary Visual Programming (NeurIPS 2024 · similarity 0.83)
- Bandit Guided Submodular Curriculum for Adaptive Subset Selection (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
