Confiabilidade e interpretação dos modelos de aprendizado de máquina em ambientes virtuais de aprendizagem com análise de desempenho e conformal prediction

Abstract

Virtual Learning Environments (VLEs) are educational platforms that can enhance knowledge retention across various fields, whether in Distance Education, Hybrid Learning, or even Faceto-Face Learning. Among the numerous available resources, forums enable asynchronous collaboration and communication among participants. This paper explores the characteristics, challenges, and limitations of using machine learning (ML) in VLEs and distance education forums. VLEs have become essential tools in modern education, offering flexibility and access to knowledge in a remote format. However, the integration of ML in these environments presents significant challenges, particularly related to the interpretation and depth of the discussion of results. In this context, the objective of this work is to propose an integrated strategy for building machine learning projects in virtual learning environments, in the context of distance education, which use forums as part of their activities. The research resulting from this study presents – in addition to a systematic review to identify the main characteristics of ML solutions applied in VLE forums – a case study proposing a binary text classifier to infer characteristics like autonomy based on message exchange in forums and message classification based on Freirean dialogue theory, as well as analyzing the performance of two text encoding methods: Sentence-BERT and TF-IDF. Another case study is presented that showcases methods for applying conformal prediction in an ML project. These combined results provide a strategy and recommendations for building ML projects with replicability, more comprehensive result analyses, and reliability, based on methods well-known in the literature and with low cost of use. Through case studies and simulations, we demonstrate how applying these techniques can reveal more detailed insights and promote more effective ML use in distance education. In conclusion, this work contributes to the literature by offering viable alternatives for critical analysis of ML results in educational contexts, promoting a better understanding and utilization of these technologies in VLEs and distance education forums.