Inference for Regression with Variables Generated by AI or Machine Learning

Laura Battaglia, Timothy Christensen, Stephen Hansen and Szymon Sacher
Additional contact information
Laura Battaglia: Oxford University
Timothy Christensen: Yale University
Szymon Sacher: Meta

No 2421, Cowles Foundation Discussion Papers from Cowles Foundation for Research in Economics, Yale University

Abstract: It has become common practice for researchers to use AI-powered information retrieval algorithms or other machine learning methods to estimate variables of economic interest, then use these estimates as covariates in a regression model. We show both theoretically and empirically that naively treating AI- and ML-generated variables as ÒdataÓ leads to biased estimates and invalid inference. We propose two methods to correct bias and perform valid inference: (i) an explicit bias correction with bias-corrected confidence intervals, and (ii) joint maximum likelihood estimation of the regression model and the variables of interest. Through several applications, we demonstrate that the common approach generates substantial bias, while both corrections perform well.

Pages: 70 pages
Date: 2025-01-02
New Economics Papers: this item is included in nep-big and nep-cmp
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://cowles.yale.edu/sites/default/files/2025-01/d2421.pdf (application/pdf)

Related works:
Working Paper: Inference for Regression with Variables Generated by AI or Machine Learning (2024)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:cwl:cwldpp:2421

Ordering information: This working paper can be ordered from
Cowles Foundation, Yale University, Box 208281, New Haven, CT 06520-8281 USA
The price is None.

Access Statistics for this paper

More papers in Cowles Foundation Discussion Papers from Cowles Foundation for Research in Economics, Yale University Yale University, Box 208281, New Haven, CT 06520-8281 USA. Contact information at EDIRC.
Bibliographic data for series maintained by Brittany Ladd ().