Research map / Label-efficient and robust learning / Recognition and long-tail
Opening the Black Box: Unraveling the Classroom Dialogue Analysis (Student Abstract)
Read the paperCheck your idea against 108,967 papers
AI-extracted summary. Written by AI from the paper's full text. Numbers and claims can be wrong or missing context, so check the paper before you cite or build on it.
Applies three gradient-based xAI attribution methods (saliency, input*gradient, integrated gradients) to explain a BERT model detecting student talk moves in classroom dialogue, validated by token-deletion accuracy drops.
Problem
Deep learning models for classroom dialogue analysis (student talk move detection) are black boxes, which undermines teacher trust and adoption.
Approach
Adapts three model-agnostic gradient-based interpretation methods to a BERT-based talk move classifier that takes student utterance paired with prior context, and validates token relevance by measuring accuracy drop when highest-relevance tokens are deleted.
Key results
- BERT-based student talk move model reached F1 0.67 and accuracy 0.78 on TalkMoves test split.
- Deleting top-relevance tokens causes a larger accuracy drop than random deletion for all three methods.
- Integrated gradients is reported as the most effective interpreting method.
Measured effects reported in the paper
| What was changed | Where | Measured | Effect |
|---|---|---|---|
| Delete tokens in order of saliency relevance rather than randomly | BERT student talk-move model · TalkMoves correctly predicted utterances | accuracy | Hurt |
| Delete tokens in order of input*gradient relevance rather than randomly | BERT student talk-move model · TalkMoves correctly predicted utterances | accuracy | Hurt |
| Delete tokens in order of integrated-gradient relevance rather than randomly | BERT student talk-move model · TalkMoves correctly predicted utterances | accuracy | Hurt |
Limitations
- No cross-validation or performance optimization; the classifier is not state-of-the-art.
- Validation is limited to deletion accuracy on correctly predicted utterances with a minimum length of 10.
- Explanations are not evaluated with teachers, so trust effects are not measured.
Experimental setup
Datasets: TalkMoves
Benchmarks: TalkMoves
Metrics: accuracy F1
Baselines: random deletion ChatGPT
Builds on: integrated gradients saliency input*gradient
Closest papers by meaning
- A Semi-supervised Learning Approach with Two Teachers to Improve Breakdown Identification in Dialogues (AAAI 2022 · similarity 0.84)
- Automated Assessment of Student Self-Explanation in Code Comprehension Using Pre-Trained Language Models (AAAI 2025 · similarity 0.83)
- Conversational Learning Diagnosis via Reasoning Multi-Turn Interactive Learning (AAAI 2026 · similarity 0.82)
- From Raw Video to Pedagogical Insights: A Unified Framework for Student Behavior Analysis (AAAI 2024 · similarity 0.82)
- iLLuMinaTE: An LLM-XAI Framework Leveraging Social Science Explanation Theories Towards Actionable Student Performance Feedback (AAAI 2025 · similarity 0.81)
- A Comprehensive Analysis of the Effectiveness of Large Language Models as Automatic Dialogue Evaluators (AAAI 2024 · similarity 0.81)
Writing a paper in this area? scime draft is a free LaTeX editor with AI included, real-time co-authors and a check of your references and venue rules before you submit. It also imports OpenAI Prism projects.