How to Use Large Language Models for Empirical Legal Research

Jonathan H. Choi

Journal of Institutional and Theoretical Economics (JITE), 2024, vol. 180, issue 2, 214-233

Abstract: Legal scholars have long annotated cases by hand to summarize and learn about developments in jurisprudence. Dramatic recent improvements in the performance of large language models (LLMs) now provide a potential alternative. This article demonstrates how to use LLMs to analyze legal documents. It evaluates best practices and suggests both the uses and potential limitations of LLMs in empirical legal research. In a simple classification task involving Supreme Court opinions, it finds that GPT-4 performs approximately as well as human coders and significantly better than a variety of prior-generation natural language processing (NLP) classifiers, with no improvement from supervised training, finetuning, or specialized prompting.

Keywords: large language models; LLMs; artificial intelligence; AI; empirical legal studies; computational analysis of law; natural language processing; machine learning JEL classification (search for similar items in EconPapers)
JEL-codes: K00 (search for similar items in EconPapers)
Date: 2024
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.mohrsiebeck.com/en/article/how-to-use- ... 101628jite-2024-0006
Fulltext access is included for subscribers to the printed version.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:mhr:jinste:urn:doi:10.1628/jite-2024-0006

Ordering information: This journal article can be ordered from
Mohr Siebeck GmbH & Co. KG, P.O.Box 2040, 72010 Tübingen, Germany

DOI: 10.1628/jite-2024-0006

Access Statistics for this article

Journal of Institutional and Theoretical Economics (JITE) is currently edited by Gerd Mühlheußer and Bayer, Ralph-C

More articles in Journal of Institutional and Theoretical Economics (JITE) from Mohr Siebeck, Tübingen
Bibliographic data for series maintained by Thomas Wolpert ().