Research map / Multi-agent systems and games / Human-AI and goal-conditioned
Play to Grade: Testing Coding Games as Classifying Markov Decision Process
Allen Nie, Emma Brunskill, Christopher J Piech
Read the paperCheck your idea against 137,217 papers
AI-extracted. The measured effects below were read by AI from the paper's full text (no summary yet). Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Use transition/reward prediction with HoareLSTM instead of Noisy Supervised Learning | Car-Easy bug-state classification · Car-Easy, single-direction agent | accuracy: 91.0 ± 4.7 -> 99.3 ± 1.9 | Improved |
| Use transition/reward prediction with HoareLSTM instead of Noisy Supervised Learning | Car-Easy bug-state classification · Car-Easy, random agent | accuracy: 60.3 ± 11.8 -> 98.5 ± 1.1 | Improved |
| Use Contrastive HoareLSTM instead of code-as-text classification | Bounce student programs · Bounce body, balanced | accuracy: 74.2 -> 93.4 | Improved |
| Use Contrastive HoareLSTM instead of code-as-text classification | Bounce student programs · Bounce tail, balanced | accuracy: 68.4 -> 94.0 | Improved |
| Add collaborative training, iteration 1, to HoareLSTM (ablation) | Car-Hard bug-state classification · Car-Hard | recall: 80.0 -> 100.0 | Improved |
| Add collaborative training, iteration 1, to Contrastive HoareLSTM (ablation) | Car-Hard bug-state classification · Car-Hard | recall: 87.1 -> 94.9 | Improved |
| Continue collaborative training from iteration 1 to iteration 2 with HoareLSTM (ablation) | Car-Hard bug-state classification · Car-Hard | accuracy: 100.0 -> 99.3 | Mixed |
| Continue collaborative training from iteration 1 to iteration 2 with Contrastive HoareLSTM (ablation) | Car-Hard bug-state classification · Car-Hard | accuracy: 97.2 -> 97.9 | Improved |
| Increase provided bug examples from 3 to 10 (ablation) | Contrastive HoareLSTM · Bounce body, balanced | accuracy: 50.0 -> 93.4 | Improved |
| Increase provided bug examples from 3 to 10 (ablation) | Contrastive HoareLSTM · Bounce tail, balanced | accuracy: 50.0 -> 94.0 | Improved |
Closest papers by meaning
- Do Coding Agents Deceive Us? Detecting and Preventing Cheating via Capped Evaluation with Randomized Tests (NeurIPS 2026 · similarity 0.85)
- EvoCodeBench: Evaluating Coding Agents in Multi-Turn Iterative Interactions (NeurIPS 2026 · similarity 0.84)
- RLEF: Grounding Code LLMs in Execution Feedback with Reinforcement Learning (ICML 2025 · similarity 0.84)
- Lessons Learned: A Multi-Agent Framework for Code LLMs to Learn and Improve (NeurIPS 2025 · similarity 0.84)
- FactorSim: Generative Simulation via Factorized Representation (NeurIPS 2024 · similarity 0.84)
- CURE: Co-Evolving Coders and Unit Testers via Reinforcement Learning (NeurIPS 2025 · similarity 0.83)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.
