Economic Behavior of Information Acquisition: Impact on Peer Grading in Massive Open Online Courses

Onesun Steve Yoo () and Dongyuan Zhan ()
Additional contact information
Onesun Steve Yoo: UCL School of Management, University College London, London E14 5AB, United Kingdom
Dongyuan Zhan: UCL School of Management, University College London, London E14 5AB, United Kingdom

Operations Research, 2023, vol. 71, issue 4, 1277-1297

Abstract: A critical issue in operating massive open online courses (MOOCs) is the scalability of providing feedback. Because it is not feasible for instructors to grade a large number of students’ assignments, MOOCs use peer grading systems. This study investigates the efficacy of that practice when student graders are rational economic agents. We characterize grading as a process of (a) acquiring information to assess an assignment’s quality and (b) reporting a score. This process entails a tradeoff between the cost of acquiring information and the benefits of accurate grading. Because the true quality is not observable, any measure of inaccuracy must reference the actions of other graders, which motivates student graders to behave strategically. We present the unique equilibrium information level and reporting strategy of a homogeneous group of student graders and then examine the outcome of peer grading. We show how both the peer grading structure and the nature of MOOC courses affect peer grading accuracy, and we identify conditions under which the process fails. There is a systematic grading bias toward the mean, which discourages students from learning. To improve current practice, we introduce a scale-shift grading scheme, theoretically examine how it can improve grading accuracy and adjust grading bias and discuss how it can be practically implemented.

Keywords: Special Issue Honoring Kenneth Arrow: Mathematical Models of Individual and Group Decision Making in Operations Research; MOOCs; peer grading; information acquisition; equilibrium grading behavior (search for similar items in EconPapers)
Date: 2023
References: Add references at CitEc
Citations:

Downloads: (external link)
http://dx.doi.org/10.1287/opre.2021.2131 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:oropre:v:71:y:2023:i:4:p:1277-1297

Access Statistics for this article

More articles in Operations Research from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().