The Behavioral Signature of GenAI in Scientific Communication

Nikos Askitas

No 18062, IZA Discussion Papers from Institute of Labor Economics (IZA)

Abstract: We examine the uptake of GPT-assisted writing in economics working paper abstracts. Using data from the IZA DP series, we detect a clear stylistic shift after the release of ChatGPT-3.5 in March 2023. This shift is evident in core textual metrics—mean word length, type-token ratio, and readability—and reflects growing convergence with machine-generated writing. While the ChatGPT launch was an exogenous shock, adoption is endogenous: authors choose whether to use AI. To capture this behavioral response, we combine stylometric analysis, machine learning classification, and prompt-based similarity testing. Event-study regressions with fixed effects and placebo checks confirm that the change is abrupt, persistent, and not explained by pre-existing trends. A similarity experiment using OpenAI’s API shows that post-ChatGPT abstracts resemble their GPT-optimized versions more closely than pre-ChatGPT resemble theirs. A classifier, trained on these variants, flags a growing share of post-March 2023 texts as GPT-like. Rather than suggesting full automation, our findings indicate selective human–AI augmentation. Our framework generalizes to other contexts such as e.g. resumes, job ads, legal briefs, research proposals, or programming code.

Keywords: AI-assisted writing; linguistic metrics; event study; machine learning; natural language processing (NLP); text analysis; academic writing; GPT adoption; diffusion of technology (search for similar items in EconPapers)
JEL-codes: C55 C81 C88 J24 L86 O33 (search for similar items in EconPapers)
Date: 2025-08
New Economics Papers: this item is included in nep-ain
References: Add references at CitEc
Citations:

Downloads: (external link)
https://docs.iza.org/dp18062.pdf (application/pdf)

Related works:
Working Paper: The Behavioral Signature of GenAI in Scientific Communication (2025)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:iza:izadps:dp18062

Ordering information: This working paper can be ordered from
IZA, Margard Ody, P.O. Box 7240, D-53072 Bonn, Germany

Access Statistics for this paper

More papers in IZA Discussion Papers from Institute of Labor Economics (IZA) IZA, P.O. Box 7240, D-53072 Bonn, Germany. Contact information at EDIRC.
Bibliographic data for series maintained by Holger Hinte ().