Research map / Multi-agent systems and games / LLM agents

BEAGLE: Behavior-Enforced Agent for Grounded Learner Emulation

NeurIPS 2026 · Multi-agent systems and games › LLM agents · approach: Agent data synthesis and behavior studies · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Proposes BEAGLE, a training-free neuro-symbolic framework that simulates novice programming students via a semi-Markov SRL controller, BKT with explicit flaw injection, and a Strategist/Executor LLM split, validated on physics coding tasks with a human Turing test.

Problem

LLM-based student simulators exhibit competency bias: they produce efficient, correct solutions rather than the iterative, error-prone struggle of novice learners, which limits their use for education research.

Approach

Combines symbolic control (semi-Markov metacognitive/cognitive dynamics fit to real logs, BKT with blocked knowledge components, observation filtering) with a decoupled Strategist/Executor LLM pipeline so the agent cannot silently correct its own intended errors.

Key results

• BEAGLE achieves D_KL = 0.31 versus best baseline 0.53 and error recurrence of 86.2% on Particle Simulator.
• In a human Turing test (N=71), classification accuracy was 52.8%, with d' = 0.15 and TOST p = 0.038 for equivalence to real traces.
• Removing the semi-Markov controller raises D_KL to 6.81; merging Strategist/Executor reduces error recurrence from 86.2% to 65.3%.

Surprising findings

• Vanilla LLM recurrence is only 7.8% while SimStudent saturates at 92%, so rigid rule-based errors can exceed human-like recurrence.
• Participants were worse than chance on AI traces (43.4%) and showed a bias toward labeling traces as real; evaluator expertise gave no advantage.

Limitations

• Residual competency bias on the strongest backbones.
• Knowledge-fluency trade-off; mastery representations may need to be richer.
• Single-agent scope; no multi-agent classroom simulation.

Experimental setup

Datasets: Snyder corpus Cohn block-based study Python pilot study Bielefeld dataset DCU dataset traces

Benchmarks: Particle Simulator Bouncing Ball Inclined Plane Gradient Descent

Metrics: solve rate steps-to-solve KL divergence debug divergence nonlinearity error recurrence rate reaction lag realism score SDT d-prime classification accuracy

Baselines: Vanilla CoT Few-Shot SimStudent LLMSS CoderAgent

Builds on: SimStudent Bayesian Knowledge Tracing Self-Regulated Learning (Winne and Hadwin) CoderAgent LLM-SS

Closest papers by meaning

• Can LLMs Reliably Simulate Human Learner Actions? A Simulation Authoring Framework for Open-Ended Learning Environments (AAAI 2025 · similarity 0.88)
• Simulating Students or Sycophantic Problem Solving? On Misconception Faithfulness of LLM Simulators (NeurIPS 2026 · similarity 0.87)
• CogBench: a large language model walks into a psychology lab (ICML 2024 · similarity 0.85)
• EduMirror: Modeling Educational Social Dynamics with Value-driven Multi-agent Simulation (ICML 2026 · similarity 0.85)
• AdaCuRL: Adaptive Curriculum Reinforcement Learning with Invalid Sample Mitigation and Historical Revisiting (AAAI 2026 · similarity 0.85)
• LLM-as-a-Tutor: Policy-Aware Prompt Adaptation for Non-Verifiable RL (NeurIPS 2026 · similarity 0.84)