Experimenting with Generative AI: Does ChatGPT Really Increase Everyone’s Productivity?

Voraprapa Nakavachara, Tanapong Potipiti and Thanee Chaiwat

No 229, PIER Discussion Papers from Puey Ungphakorn Institute for Economic Research

Abstract: Generative AI technologies such as ChatGPT, Gemini, and MidJourney have made remarkable progress in recent years. Recent literature has documented ChatGPT’s positive impact on productivity in areas where it has strong expertise—attributable to extensive training datasets—such as the English language and Python/SQL programming. However, the literature is still limited regarding ChatGPT’s performance in areas where its capabilities could still be further enhanced. In this paper, we asked participants to perform writing analysis tasks in a non-English language (specifically, Thai) and math & data analysis tasks using a less frequently used programming package (specifically Stata). The findings suggest that, on average, participants performed better using ChatGPT in terms of scores and time taken to complete the tasks. However, a detailed examination reveals that 34% of participants saw no improvement in writing analysis tasks, and 42% did not improve in math & data analysis tasks when employing ChatGPT. Further investigation indicated that higher-ability participants, as proxied by their econometrics grades, were the ones who performed worse in writing analysis tasks when using ChatGPT. We also found evidence that participants with better digital skills performed better with ChatGPT. This research provides insights on the impact of generative AI. Thus, relevant parties can make informed decisions regarding appropriate strategies, policies, and educational systems. It also highlights the critical role of human skills in addressing and complementing the limitations of AI.

Keywords: ChatGPT; Generative AI; Large Language Models; Labor Productivity (search for similar items in EconPapers)
JEL-codes: A20 D24 J24 O33 (search for similar items in EconPapers)
Pages: 24 pages
Date: 2025-01
New Economics Papers: this item is included in nep-ain, nep-eff and nep-lma
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://www.pier.or.th/files/dp/pier_dp_229.pdf (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:pui:dpaper:229

Ordering information: This working paper can be ordered from
https://www.pier.or.th/en/dp/229/

Access Statistics for this paper

More papers in PIER Discussion Papers from Puey Ungphakorn Institute for Economic Research Contact information at EDIRC.
Bibliographic data for series maintained by ().