When Algorithms Rate Performance: Do Large Language Models Replicate Human Evaluation Biases?

Rainer Michael Rilke () and Dirk Sliwka ()
Additional contact information
Rainer Michael Rilke: WHU - Otto Beisheim School of Management
Dirk Sliwka: University of Cologne

No 384, ECONtribute Discussion Papers Series from University of Bonn and University of Cologne, Germany

Abstract: A large body of research across management, psychology, accounting, and economics shows that subjective performance evaluations are systematically biased: ratings cluster near the midpoint of scales and are often excessively lenient. As organizations increasingly adopt large language models (LLMs) for evaluative tasks, little is known about how these systems perform when assessing human performance. We document that, in the absence of clear objective standards and when individuals are rated independently, LLMs reproduce the familiar patterns of human raters. However, LLMs generate greater dispersion and accuracy when evaluating multiple individuals simultaneously. With noisy but objective performance signals, LLMs provide substantially more accurate evaluations than human raters, as they (i) are less subject to biases arising from concern for the evaluated employee and (ii) make fewer mistakes in information processing closely approximating rational Bayesian benchmarks.

Keywords: Performance Evaluation; Large Language Models; Signal Objectivity; Algorithmic Judgment; Gen-AI (search for similar items in EconPapers)
JEL-codes: J24 J28 M12 M53 (search for similar items in EconPapers)
Pages: 27 pages
Date: 2026-01
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.econtribute.de/RePEc/ajk/ajkdps/ECONtribute_384_2026.pdf First version, 2026 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:ajk:ajkdps:384

Access Statistics for this paper

More papers in ECONtribute Discussion Papers Series from University of Bonn and University of Cologne, Germany Niebuhrstrasse 5, 53113 Bonn, Germany.
Bibliographic data for series maintained by ECONtribute Office ().