Research map / Multi-agent systems and games / LLM agents

Conversational Learning Diagnosis via Reasoning Multi-Turn Interactive Learning

AAAI 2026 · Multi-agent systems and games › LLM agents · approach: Agent data synthesis and behavior studies · 0 citations (OpenAlex)

Read the paperCheck your idea against 108,967 papers

AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.

Introduces ParLD, a multi-agent preview-analyze-reason framework with a chain reflector that uses GPT-4.1/GPT-4o to diagnose students' evolving cognitive state over multi-turn tutoring dialogues.

Problem

Conversational learning diagnosis: inferring students' latent, evolving knowledge-concept mastery across multi-turn tutor-student dialogues, which existing KT/CDM methods and direct LLM prompting handle poorly.

Approach

A prompt-based agent chain (Behavior Previewer producing a ZPD-based behavior schema, State Analyzer updating KC mastery, Performance Reasoner predicting outcome) with a Chain Reflector that compares predictions to observed outcomes and reruns analysis via conversation memory.

Key results

• ParLD (GPT-4.1) reaches 68.72 ACC and 66.15 F1 on MathDial, and 57.26 ACC on CoMTA, outperforming DKT, AKT, DKVMN, SAINT, and SimpleKT.
• In tutoring simulation on MathDial, ParLD with GPT-4.1 yields a 72.22% correct rate versus 56.48% for Direct Respond.
• Two-rater quality evaluation on 20 simulated conversations rates ParLD above Direct Analyze in accuracy (2.475 vs 2.1) and coherence (2.7 vs 2.425) on a 3-point scale.

Surprising findings

• Removing the Behavior Previewer (w/o P) yields the worst prediction performance, and reflection alone on the analyze-reason chain can mislead the analyzer.

Limitations

• Relies on proprietary API LLMs with temperature 0, limiting reproducibility and cost-efficiency.
• Ground-truth cognitive states are latent; validation relies on proxy tasks (performance prediction, simulated tutoring) and a small two-rater human study.
• Reflection iterations capped for cost reasons, and CoMTA subset is small (116 conversations).

Experimental setup

Datasets: MathDial CoMTA

Benchmarks: MathDial CoMTA

Metrics: ACC F1 Correct Rate Avg. Turns Intersection Avg. Turns Accuracy (human rating) Coherence (human rating)

Baselines: DKT AKT DKVMN SAINT SimpleKT Direct Respond Direct Analyze

Builds on: Knowledge Tracing Zone of Proximal Development MathDial

Closest papers by meaning

• EduDiag: A Benchmark for Educational Diagnostic Reasoning with Error Tracing and Correction on Large Multimodal Models (CVPR 2026 · similarity 0.86)
• Multi-Perspective Consolidation Enhanced Cognitive Diagnosis via Conditional Diffusion Model (AAAI 2025 · similarity 0.85)
• Collaborative Cognitive Diagnosis with Disentangled Representation Learning for Learner Modeling (NeurIPS 2024 · similarity 0.85)
• Knowledge Is Power: Harnessing Large Language Models for Enhanced Cognitive Diagnosis (AAAI 2025 · similarity 0.85)
• From EduVisBench to EduVisAgent: A Benchmark and Multi-Agent Framework for Reasoning-Driven Pedagogical Visualization (ICLR 2026 · similarity 0.84)
• iLLuMinaTE: An LLM-XAI Framework Leveraging Social Science Explanation Theories Towards Actionable Student Performance Feedback (AAAI 2025 · similarity 0.84)