Large Language Models: An Applied Econometric Framework

Jens Ludwig, Sendhil Mullainathan and Ashesh Rambachan

No 33344, NBER Working Papers from National Bureau of Economic Research, Inc

Abstract: Large language models (LLMs) enable researchers to analyze text at unprecedented scale and minimal cost. Researchers can now revisit old questions and tackle novel ones with rich data. We provide an econometric framework for realizing this potential in two empirical uses. For prediction problems – forecasting outcomes from text – valid conclusions require “no training leakage” between the LLM’s training data and the researcher’s sample, which can be enforced through careful model choice and research design. For estimation problems – automating the measurement of economic concepts for downstream analysis – valid downstream inference requires combining LLM outputs with a small validation sample to deliver consistent and precise estimates. Absent a validation sample, researchers cannot assess possible errors in LLM outputs, and consequently seemingly innocuous choices (which model, which prompt) can produce dramatically different parameter estimates. When used appropriately, LLMs are powerful tools that can expand the frontier of empirical economics.

JEL-codes: C01 C45 (search for similar items in EconPapers)
Date: 2025-01
New Economics Papers: this item is included in nep-big and nep-cmp
Note: AP CF CH DAE DEV ED EEE EFG EH LE LS PE POL PR TWP
References: Add references at CitEc
Citations: View citations in EconPapers (8)

Downloads: (external link)
http://www.nber.org/papers/w33344.pdf (application/pdf)
Access to the full text is generally limited to series subscribers, however if the top level domain of the client browser is in a developing country or transition economy free access is provided. More information about subscriptions and free access is available at http://www.nber.org/wwphelp.html. Free access is also available to older working papers.

Related works:
Working Paper: Large Language Models: An Applied Econometric Framework (2025)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:nbr:nberwo:33344

Ordering information: This working paper can be ordered from
http://www.nber.org/papers/w33344
The price is Paper copy available by mail.

Access Statistics for this paper

More papers in NBER Working Papers from National Bureau of Economic Research, Inc National Bureau of Economic Research, 1050 Massachusetts Avenue Cambridge, MA 02138, U.S.A.. Contact information at EDIRC.
Bibliographic data for series maintained by ().