Do humans identify AI-generated text better than machines? Evidence based on excerpts from German theses☆

Alexandra Fiedler and Jörg Döpke

International Review of Economics Education, 2025, vol. 49, issue C

Abstract: We investigate whether human experts can identify AI-generated academic texts more accurately than current machine-based detectors. Conducted as a survey experiment at a German university of applied sciences, 63 lecturers in engineering, economics, and social sciences were asked to evaluate short excerpts (200–300 words) from both human-generated and AI-generated texts. These texts varied by discipline and writing level (student vs. professional) with the AI-generated content. The results show that both human evaluators and AI detectors correctly identified AI-generated texts only slightly better than chance, with humans achieving a recognition rate of 57 % for AI texts and 64 % for human-generated texts. There was no statistically significant difference between human and machine performance. Notably, professional-level AI texts were the most difficult to identify, with less than 20 % of respondents correctly classifying them. Regression analyses suggest that prior teaching experience slightly improves recognition accuracy, while subjective judgments of text quality were not influenced by actual or presumed authorship. These findings suggest that current written examination practices are increasingly vulnerable to undetected AI use. Both human judgment and existing AI detectors show high error rates, especially for high-quality AI-generated content. We conclude that a reconsideration of traditional assessment formats in academia is warranted.

Keywords: Artificial intelligence; Written examinations; Grading (search for similar items in EconPapers)
JEL-codes: A2 C88 I23 (search for similar items in EconPapers)
Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S1477388025000131
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:ireced:v:49:y:2025:i:c:s1477388025000131

DOI: 10.1016/j.iree.2025.100321

Access Statistics for this article

International Review of Economics Education is currently edited by Guest, Ross

More articles in International Review of Economics Education from Elsevier
Bibliographic data for series maintained by Catherine Liu ().