Human intelligence versus artificial intelligence in classifying economics research articles: exploratory evidence

Jussi Heikkilä

EconStor Open Access Articles and Book Chapters, 2024, vol. 81, issue 7, 18-30

Abstract: Purpose We compare human intelligence to artificial intelligence (AI) in the choice of appropriate Journal of Economic Literature (JEL) codes for research papers in economics. Design/methodology/approach We compare the JEL code choices related to articles published in the recent issues of the Journal of Economic Literature and the American Economic Review and compare these to the original JEL code choices of the authors in earlier working paper versions and JEL codes recommended by various generative AI systems (OpenAI’s ChatGPT, Microsoft’s Copilot, Google’s Gemini) based on the abstracts of the articles. Findings There are significant discrepancies and often limited overlap between authors’ choices of JEL codes, editors’ choices as well as the choices by contemporary widely used AI systems. However, the observations suggest that generative AI can augment human intelligence in the micro-task of choosing the JEL codes and, thus, save researchers time. Research limitations/implications Rapid development of AI systems makes the findings quickly obsolete. Practical implications AI systems may economize on classification costs and (semi-)automate the choice of JEL codes by recommending the most appropriate ones. Future studies may apply the presented approach to analyze whether the JEL code choices between authors, editors and AI systems converge and become more consistent as humans increasingly interact with AI systems. Originality/value We assume that the choice of JEL codes is a micro-task in which boundedly rational decision-makers rather satisfice than optimize. This exploratory experiment is among the first to compare human intelligence and generative AI in choosing and justifying the choice of optimal

Keywords: Artificial intelligence; Large language models; Search costs; Bounded rationality (search for similar items in EconPapers)
JEL-codes: A11 A14 C88 D81 D83 (search for similar items in EconPapers)
Date: 2024
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
https://www.econstor.eu/bitstream/10419/307995/1/H ... man-intelligence.pdf (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:zbw:espost:307995

DOI: 10.1108/JD-05-2024-0104

Access Statistics for this article

More articles in EconStor Open Access Articles and Book Chapters from ZBW - Leibniz Information Centre for Economics Contact information at EDIRC.
Bibliographic data for series maintained by ZBW - Leibniz Information Centre for Economics ().