Do AI Occupational-Exposure Scores Measure AI? AIOE and Eloundou (2024) Largely Capture Cognitive Content; Webb (2020) Does Not

Sudhanshu Rai

MPRA Paper from University Library of Munich, Germany

Abstract: A growing empirical literature uses pre-built "AI occupational exposure" scores (most prominently the AI Occupational Exposure (AIOE) index of Felten, Raj, and Seamans (2021) and the GPT-4 task-exposure measure of Eloundou et al. (2024)) as occupation-level treatments or predictors for AI's labor-market effects. We show that two of the most-cited scores, AIOE and Eloundou's GPT-4 measure, substantially re-label cognitive task content rather than capturing AI-specific exposure, a construct-validity problem that does not extend to a third, differently-built score (Webb 2020, patent-based). This note horse-races these three measures against transparent cognitive/manual task-content indices and the established Autor–Dorn Routine Task Intensity (RTI) measure. Across 773 occupations: (i) the ten-plus AI "applications" underlying AIOE collapse to a single factor (first principal component ≈ 88%); (ii) AIOE and Eloundou each correlate strongly with a cognitive-ability index (+0.85 / +0.70) and negatively with a manual-ability index (−0.91 / −0.83), correlate 0.86 with each other, but only moderately with RTI (−0.33 / −0.30): the confound is specifically cognitive ability level, not the classic routine-task polarization axis; (iii) each score's positive wage association reverses sign controlling for cognitive content but is barely affected by controlling for RTI; and (iv) the much-cited pre-ChatGPT "AI foresight" wage-divergence pattern collapses to near-zero under the cognitive control (not under the RTI control). Critically, this collapse is not universal: Webb's patent-text-overlap score is essentially uncorrelated with AIOE (r=0.03) and Eloundou (r=−0.03), only weakly related to cognitive content (r=0.13), and its modest wage associations do not reverse under cognitive control. The cognitive-content collapse is a signature of how a score is built: subjective crowd-relatedness ratings (AIOE) or LLM/human task judgments (Eloundou), not an inherent property of occupational AI-exposure measurement. Studies using relatedness- or judgment-based exposure scores should control for cognitive content and re-interpret accordingly; a patent-based measure is not shown here to have the same problem, though its own construct validity is untested.

Keywords: AI occupational exposure; Felten-Raj-Seamans index; GPT-4 task exposure; construct validity; task content measures; routine task intensity; Autor-Dorn RTI; patent-based automation exposure (search for similar items in EconPapers)
JEL-codes: C38 C52 C81 J24 O33 (search for similar items in EconPapers)
Date: 2026-07-05
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://mpra.ub.uni-muenchen.de/129904/1/MPRA_paper_129904.pdf original version (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:pra:mprapa:129904

Access Statistics for this paper

More papers in MPRA Paper from University Library of Munich, Germany Ludwigstraße 33, D-80539 Munich, Germany. Contact information at EDIRC.
Bibliographic data for series maintained by Joachim Winter ().