Human Realignment: An Empirical Study of LLMs as Legal Decision-Aids in Moral Dilemmas

Christoph Engel (), Yoan Hermstrüwer () and Alison Kim ()
Additional contact information
Christoph Engel: Max Planck Institute for Research on Collective Goods, Bonn
Yoan Hermstrüwer: University of Zurich
Alison Kim: University of Zurich

No 2025_03, Discussion Paper Series of the Max Planck Institute for Research on Collective Goods from Max Planck Institute for Research on Collective Goods

Abstract: Recent advances in AI create possibilities for delegating legal decision-making to machines or enhancing human adjudication through AI assistance. Using classic normative conflicts-the trolley problem and similar moral dilemmas-as a proof of concept, we examine the alignment between AI legal reasoning and human judgment. In our baseline experiment, we find a pronounced mismatch between decisions made by GPT and those of human subjects. This misalignment raises substantive concerns for AI-powered legal decision-aids. We investigate whether explicit normative guidance can address this misalignment, with mixed results. GPT-3.5 is susceptible to such intervention, but frequently refuses to decide when faced with a moral dilemma. GPT-4 is outright utilitarian, and essentially ignores the instruction to decide on deontological grounds. GPT-o3-mini faithfully implements this instruction, but is unwilling to balance deontological and utilitarian concerns if instructed to do so. At least for the time being, explicit normative instructions are not fully able to realign AI advice with the normative convictions of the legislator.

Keywords: large language models; human-AI alignment; rule of law; moral dilemmas; trolley problems (search for similar items in EconPapers)
JEL-codes: C99 D63 D81 K10 K40 Z13 (search for similar items in EconPapers)
Date: 2025-04
New Economics Papers: this item is included in nep-ain and nep-exp
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.coll.mpg.de/pdf_dat/2025_03online.pdf (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:mpg:wpaper:2025_03

Access Statistics for this paper

More papers in Discussion Paper Series of the Max Planck Institute for Research on Collective Goods from Max Planck Institute for Research on Collective Goods Contact information at EDIRC.
Bibliographic data for series maintained by Marc Martin ().