Classical Test Theory and Item Response Theory in Measuring Validity of Peer-Grading in Massive Open Online Courses

Daria Kravchenko
Additional contact information
Daria Kravchenko: http://www.hse.ru/en/org/persons/194964801

Voprosy obrazovaniya / Educational Studies Moscow, 2018, issue 4, 99-115

Abstract: Daria Kravchenko - Analyst, Centre for Psychometrics in eLearning, National Research University Higher School of Economics. E-mail: dakravchenko@hse.ruThe article presents the results of research on validity of peer-review assignments in massive open online courses within the framework of classical test theory (CTT) and item response theory (IRT). CTT-based analysis yielded data on convergent validity of the peer-review assignment, the low level of its criterion validity, and rater disagreement. IRT-based analysis revealed rater bias and established that experts largely tend to be lenient and overrate their peers. The findings are used to discuss the advantages and disadvantages of the psychometric theories in question and the opportunities for combining the two.

Keywords: massive open online courses; peer grading; classical test theory; item response theory; peer-review assignments (search for similar items in EconPapers)
Date: 2018
References: Add references at CitEc
Citations:

Downloads: (external link)
https://vo.hse.ru/data/2018/12/15/1144780536/05%20Kravchenko.pdf (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:nos:voprob:2018:i:4:p:99-115

Access Statistics for this article

More articles in Voprosy obrazovaniya / Educational Studies Moscow from National Research University Higher School of Economics
Bibliographic data for series maintained by Marta Morozova ().