A Technological Construction of Society comparing GPT-4 and human respondents for occupational evaluation in the UK

Gmyrek, Pawel,, Lutz, Christoph, and Newlands, Gemma,

ILO Working Papers from International Labour Organization

Abstract: Despite initial research about the biases and perceptions of Large Language Models (LLMs), we lack evidence on how LLMs evaluate occupations, especially in comparison to human evaluators. In this paper, we present a systematic comparison of occupational evaluations by GPT-4 with those from an in-depth, high-quality and recent human respondents survey in the United Kingdom. Covering the full ISCO-08 occupational landscape, with 580 occupations and two distinct metrics (prestige and social value), our findings indicate that GPT-4 and human scores are highly correlated across all ISCO-08 major groups. In absolute terms, GPT-4 scores are more generous than those of the human respondents. At the same time, GPT-4 substantially under or overestimates the occupational prestige and social value of many occupations, particularly for emerging digital and stigmatized occupations.

Keywords: occupational classification; occupational qualification; artificial intelligence; survey (search for similar items in EconPapers)
Pages: 1 online resource (50 p.) pages
Date: 2024
New Economics Papers: this item is included in nep-ain and nep-cmp
References: Add references at CitEc
Citations: View citations in EconPapers (1)

Published in ILO working paper series

Downloads: (external link)
https://doi.org/10.54394/UQOQ5153 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:ilo:ilowps:995347793502676

Access Statistics for this paper

More papers in ILO Working Papers from International Labour Organization Contact information at EDIRC.
Bibliographic data for series maintained by Vesa Sivunen ().