Programming knowledge tracing based on knowledge concept identification and hierarchical modeling

Programming knowledge tracing (PKT) aims to evaluate students’ mastery of knowledge concepts and predict their future performance based on datasets containing question–answer (code) instances. However, each instance is only labelled with a few knowledge concepts, and lacks all related knowledge concepts, as well as procedural and hierarchical relations among them. This leads to insufficient utilization of programming process information and hinders accurate evaluation of students’ programming mastery at a fine granularity from a systematic perspective. To address this problem, we propose PKT-KCIHM, a PKT method based on knowledge concept identification and hierarchical modeling. Specifically, we design a Tree-of-Thoughts-inspired self-verifying knowledge concept identification algorithm to recognize the relations of question-to-knowledge concept and answer-to-knowledge concept. Based on above relations, we construct a hierarchical graph for programming courses, and design a dual-dimensional LSTM network to capture students’ knowledge states. Finally, we propose a joint loss function that adds a mastery-constrained loss to the knowledge tracing prediction loss. Extensive experimental results on three datasets demonstrate the effectiveness of PKT-KCIHM.