Evaluation of an Algorithm for Automatic Grading of Forum Messages in MOOC Discussion Forums

Raquel L. Pérez-Nicolás, Carlos Alario-Hoyos, Iria Estévez-Ayres, Pedro Manuel Moreno-Marcos, Pedro J. Muñoz-Merino and Carlos Delgado Kloos
Additional contact information
Raquel L. Pérez-Nicolás: Department of Telematics Engineering, Universidad Carlos III de Madrid, E-28911 Leganés, Spain
Carlos Alario-Hoyos: Department of Telematics Engineering, Universidad Carlos III de Madrid, E-28911 Leganés, Spain
Iria Estévez-Ayres: Department of Telematics Engineering, Universidad Carlos III de Madrid, E-28911 Leganés, Spain
Pedro Manuel Moreno-Marcos: Department of Telematics Engineering, Universidad Carlos III de Madrid, E-28911 Leganés, Spain
Pedro J. Muñoz-Merino: Department of Telematics Engineering, Universidad Carlos III de Madrid, E-28911 Leganés, Spain
Carlos Delgado Kloos: Department of Telematics Engineering, Universidad Carlos III de Madrid, E-28911 Leganés, Spain

Sustainability, 2021, vol. 13, issue 16, 1-18

Abstract: Discussion forums are a valuable source of information in educational platforms such as Massive Open Online Courses (MOOCs), as users can exchange opinions or even help other students in an asynchronous way, contributing to the sustainability of MOOCs even with low interaction from the instructor. Therefore, the use of the forum messages to get insights about students’ performance in a course is interesting. This article presents an automatic grading approach that can be used to assess learners through their interactions in the forum. The approach is based on the combination of three dimensions: (1) the quality of the content of the interactions, (2) the impact of the interactions, and (3) the user’s activity in the forum. The evaluation of the approach compares the assessment by experts with the automatic assessment obtaining a high accuracy of 0.8068 and Normalized Root Mean Square Error (NRMSE) of 0.1799, which outperforms previous existing approaches. Future research work can try to improve the automatic grading by the training of the indicators of the approach depending on the MOOCs or the combination with text mining techniques.

Keywords: MOOC; discussion forum; automatic grading; evaluation; interactions; quality; impact (search for similar items in EconPapers)
JEL-codes: O13 Q Q0 Q2 Q3 Q5 Q56 (search for similar items in EconPapers)
Date: 2021
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (2)

Downloads: (external link)
https://www.mdpi.com/2071-1050/13/16/9364/pdf (application/pdf)
https://www.mdpi.com/2071-1050/13/16/9364/ (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:gam:jsusta:v:13:y:2021:i:16:p:9364-:d:618500

Access Statistics for this article

Sustainability is currently edited by Ms. Alexandra Wu

More articles in Sustainability from MDPI
Bibliographic data for series maintained by MDPI Indexing Manager ().