Man vs Machine: Can AI Grade and Give Feedback Like a Human?

Arnaud Chevalier (), Jakub Orzech and Petar Stankov
Additional contact information
Arnaud Chevalier: Royal Holloway, University of London
Jakub Orzech: University of London

No 17511, IZA Discussion Papers from Institute of Labor Economics (IZA)

Abstract: Grading and providing feedback are two of the most time-consuming activities in education. We developed a randomised controlled trial (RCT) to test whether they could be performed by generative artificial intelligence (Gen-AI). We randomly allocated undergraduate students to feedback provided either by a human instructor, ChatGPT 3.5, or ChatGPT 4. Our results show that: (i) Students treated with the freely accessible ChatGPT 3.5 received lower grades in subsequent assessments than their peers in the control group who always received human feedback; (ii) No such penalty was observed for ChatGPT 4. Separately, we tested the capacity of Gen-AI to grade student work. Gen-AI grades and ranks were significantly different than human-generated grades. Overall, while the newest LLM helps learning as well as a human, its ability to grade student work is still inferior.

Keywords: feeback; grading; Artificial Intelligence; learning with Gen-AI (search for similar items in EconPapers)
JEL-codes: A22 C93 I23 I24 (search for similar items in EconPapers)
Pages: 49 pages
Date: 2024-12
New Economics Papers: this item is included in nep-ain, nep-cmp and nep-exp
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://docs.iza.org/dp17511.pdf (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:iza:izadps:dp17511

Ordering information: This working paper can be ordered from
IZA, Margard Ody, P.O. Box 7240, D-53072 Bonn, Germany

Access Statistics for this paper

More papers in IZA Discussion Papers from Institute of Labor Economics (IZA) IZA, P.O. Box 7240, D-53072 Bonn, Germany. Contact information at EDIRC.
Bibliographic data for series maintained by Holger Hinte ().