Predicting Skill Shortages in Labor Markets: A Machine Learning Approach

Nik Dawson, Marian-Andrei Rizoiu, Benjamin Johnston and Mary-Anne Williams

Papers from arXiv.org

Abstract: Skill shortages are a drain on society. They hamper economic opportunities for individuals, slow growth for firms, and impede labor productivity in aggregate. Therefore, the ability to understand and predict skill shortages in advance is critical for policy-makers and educators to help alleviate their adverse effects. This research implements a high-performing Machine Learning approach to predict occupational skill shortages. In addition, we demonstrate methods to analyze the underlying skill demands of occupations in shortage and the most important features for predicting skill shortages. For this work, we compile a unique dataset of both Labor Demand and Labor Supply occupational data in Australia from 2012 to 2018. This includes data from 7.7 million job advertisements (ads) and 20 official labor force measures. We use these data as explanatory variables and leverage the XGBoost classifier to predict yearly skills shortage classifications for 132 standardized occupations. The models we construct achieve macro-F1 average performance scores of up to 83 per cent. Our results show that job ads data and employment statistics were the highest performing feature sets for predicting year-to-year skills shortage changes for occupations. We also find that features such as 'Hours Worked', years of 'Education', years of 'Experience', and median 'Salary' are highly important features for predicting occupational skill shortages. This research provides a robust data-driven approach for predicting and analyzing skill shortages, which can assist policy-makers, educators, and businesses to prepare for the future of work.

Date: 2020-04, Revised 2020-08
New Economics Papers: this item is included in nep-big
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Published in Workshop on Human-in-the-Loop Methods and Future of Work in BigData (HMData'20), 2020

Downloads: (external link)
http://arxiv.org/pdf/2004.01311 Latest version (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:arx:papers:2004.01311

Access Statistics for this paper

More papers in Papers from arXiv.org
Bibliographic data for series maintained by arXiv administrators ().