Exploring Pre-Service Teachers’ Cognitive Processes and Calibration with an Unsupervised Learning-Based Automated Evaluation System

Jiseung Yoo, Jisun Park, Minsu Ha and Chelcea Mae Lagmay Darang

SAGE Open, 2024, vol. 14, issue 3, 21582440241262864

Abstract: In the context of formative assessment in classrooms, the incorporation of automated evaluation (AE) systems and teachers’ interactions with them hold significant importance. This study aimed to investigate the cognitive processes of pre-service teachers as they engaged with an AE system. We developed an unsupervised learning-based AE system, the Scoring Assistant using Artificial Intelligence (SAAI). SAAI calculates scores without relying on predefined labels and generates scientific keywords from student responses to provide constructive feedback. We collected a substantial number of constructed responses from students, and four pre-service teachers evaluated these responses initially without any external assistance and then re-evaluated them using SAAI scores as a reference point. Employing a mixed-methods approach, this study demonstrated a strong level of consistency between human raters and SAAI scores. Pre-service teachers also reflectively recalibrated their assessments and adjusted their rubrics to identify students’ learning more accurately. This study highlights the practical application of AE in real classroom settings and demonstrates how AE can enhance efficiency and accuracy in K-12 science assessments, thus supporting teachers.

Keywords: automated evaluation; science education; formative assessment; unsupervised learning; calibration (search for similar items in EconPapers)
Date: 2024
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://journals.sagepub.com/doi/10.1177/21582440241262864 (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:sae:sagope:v:14:y:2024:i:3:p:21582440241262864

DOI: 10.1177/21582440241262864

Access Statistics for this article

More articles in SAGE Open
Bibliographic data for series maintained by SAGE Publications ().