A machine learning approach to graduate admissions and the role of letters of recommendation

Yijun Zhao, Xiaoyu Chen, Haoran Xue and Gary M Weiss

PLOS ONE, 2023, vol. 18, issue 10, 1-17

Abstract: The graduate admissions process is time-consuming, subjective, and complicated by the need to combine information from diverse data sources. Letters of recommendation (LORs) are particularly difficult to evaluate and it is unclear how much impact they have on admissions decisions. This study addresses these concerns by building machine learning models to predict admissions decisions for two STEM graduate programs, with a focus on examining the contribution of LORs in the decision-making process. We train our predictive models leveraging information extracted from structured application forms (e.g., undergraduate GPA, standardized test scores, etc.), applicants’ resumes, and LORs. A particular challenge in our study is the different modalities of application data (i.e., text vs. structured forms). To address this issue, we converted the textual LORs into features using a commercial natural language processing product and a manual rating process that we developed. By analyzing the predictive performance of the models using different subsets of features, we show that LORs alone provide only modest, but useful, predictive signals to admission decisions; the best model for predicting admissions decisions utilized both LOR and non-LOR data and achieved 89% accuracy. Our experiments demonstrate promising results in the utility of automated systems for assisting with graduate admission decisions. The findings confirm the value of LORs and the effectiveness of our feature engineering methods from LOR text. This study also assesses the significance of individual features using the SHAP method, thereby providing insight into key factors affecting graduate admission decisions.

Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0291107 (text/html)
https://journals.plos.org/plosone/article/file?id= ... 91107&type=printable (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:plo:pone00:0291107

DOI: 10.1371/journal.pone.0291107

Access Statistics for this article

More articles in PLOS ONE from Public Library of Science
Bibliographic data for series maintained by plosone ().