Documenting Differences Between Humans and AI in High-Stakes Decisions: A Labor Market Turing Test

Andres Sebastian Abril Arteaga, Marcos Rangel and Wladimir Zanoni

No 14296, IDB Publications (Working Papers) from Inter-American Development Bank

Abstract: We developed a Labor Market Turing Test (LMTT) to measure human-AI decision alignment using data from 277 human recruiters engaged in a field experiment set in Quito, Ecuador. We augmented the pool of recruiters by creating AI teams, each of which with differing impersonation of human-like traits, and compared their choices to humans and a benchmark AI model. While AI teams were more consistent, they selected candidates with a pattern that markedly different from human choices. In fact, random decisions mir- rored human choices more closely than our most human-like AI agents. These findings reveal a fundamental tension between algorithmic consistency and human judgment. That humans were closer to a random process when com- paring candidates with equal productivity might be seen as a fairer outcome. Our LMTT framework, which involves isolating and estimating a machina la- tent trait, provides a quantitative tool for assessing human-AI alignment which can be employed across critical domains, such as healthcare, justice, and edu- cation, thereby informing the design and AI governance.

Keywords: Algorithmic Fairness; Human-AI Alignment; Latent Trait Analysis (search for similar items in EconPapers)
JEL-codes: C91 J71 M51 (search for similar items in EconPapers)
Date: 2025-09
References: Add references at CitEc
Citations:

Downloads: (external link)
https://publications.iadb.org/publications/english ... rket-Turing-Test.pdf (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:idb:brikps:14296

DOI: 10.18235/0013729

Access Statistics for this paper

More papers in IDB Publications (Working Papers) from Inter-American Development Bank Contact information at EDIRC.
Bibliographic data for series maintained by Felipe Herrera Library ().