LLM as a law professor: Having a large language model write a commentary on freedom of assembly

Johannes Kruse and Christoph Engel ()
Additional contact information
Johannes Kruse: Max Planck Institute for Research on Collective Goods, Bonn
Christoph Engel: Max Planck Institute for Research on Collective Goods, Bonn

No 2025_14, Discussion Paper Series of the Max Planck Institute for Research on Collective Goods from Max Planck Institute for Research on Collective Goods

Abstract: In many jurisdictions, academia is at the service of legal practice. Law professors write commentaries that summarize the state of the art of doctrine, chiefly of jurisprudence. In the spirit of a proof of concept, using the guarantee of freedom of assembly in the European Convention on Human Rights, we show that this task can be completely outsourced to large language models. Using standard NLP metrics and an LLM as a judge approach, we develop an evaluation pipeline that works without costly human annotation. The commentaries fully written by GPT 4o, Gemini 2.5 flash or Kimi K2 Instruct are on par with their best human written competitor, the Guide provided by the Court itself.

Date: 2025-09
New Economics Papers: this item is included in nep-cmp and nep-law
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.coll.mpg.de/pdf_dat/2025_14online.pdf (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:mpg:wpaper:2025_14

Access Statistics for this paper

More papers in Discussion Paper Series of the Max Planck Institute for Research on Collective Goods from Max Planck Institute for Research on Collective Goods Contact information at EDIRC.
Bibliographic data for series maintained by Marc Martin ().