Enhancing Enhancing High School Learning Management Systems Through Explainable AI-Based Behavioral Analytics for Personalized Learning Support

Abstract

Purpose – This study develops an Explainable AI (XAI)-enhanced Learning Analytics framework for analyzing student learning behaviors in a senior high school Learning Management System (LMS) and supporting retrospective performance-risk identification using score-independent behavioral features.
Methods – The study analyzed 57,095 LMS quiz activity records from 111 students collected between July 2024 and October 2025. Seven behavioral variables were initially engineered, while Answer Accuracy, Learning Consistency, and Learning Efficiency were excluded from predictive modeling because they are derived from score-related information. The final models used Quiz Frequency, Learning Duration, Activity Level, and Dominant Study-Time Pattern. Random Forest, Decision Tree, and Logistic Regression were evaluated, with SHAP TreeExplainer used for interpretation.
Findings – Decision Tree achieved the highest accuracy (73.9%), macro F1-score (0.638), and macro ROC-AUC (0.771) on the 23-student test set. Random Forest achieved 69.6% accuracy, 0.631 macro F1-score, and 0.669 macro ROC-AUC, while Logistic Regression achieved 56.5%, 0.455, and 0.656, respectively. Learning Duration had the highest mean absolute SHAP value.
Research Implications – The results indicate that score-independent behavioral features provide a limited-to-moderate signal for retrospective performance classification. Prospective early-warning utility requires temporal validation.
Originality – The study integrates XAI with behavioral Learning Analytics while explicitly separating score-independent predictors from the score-derived target to support interpretable educational decision-making