Feasibility and Reliability of Automated Coding of Occupation in the Health and Retirement Study

Brooke Helppie-McFall and Amanda Sonnega
Additional contact information
Brooke Helppie-McFall: University of Michigan
Amanda Sonnega: University of Michigan

Working Papers from University of Michigan, Michigan Retirement Research Center

Abstract: Due to advances in computing power and the increase in coverage of longitudinal datasets in the Health and Retirement Study (HRS) that provide information about detailed occupations, demand has increased among researchers for improved occupation and industry data. The detailed data are currently hard to use because they were coded at different times, and the codeframes are, therefore, not consistent over time. Additionally, the HRS gathers new occupation and industry information from respondents every two years, and coding of new data at each wave is costly and time-consuming. In this project, we tested the NIOSH Industry and Occupation Computerized Coding System (NIOCCS) to see if it could improve processes for coding data from the HRS. We tested results from NIOCCS against results from a human coder for multiple datasets. NIOCCs does reasonably well compared to coding results from a highly-trained, professional occupation and industry coder, with kappa inter-rater reliability on detailed codes of just under 70 percent and agreement rates on broader codes of around 80 percent; however, code rates for NIOCCS for the datasets tested ranged from 60 percent to 72 percent, as compared to a professional coder’s ability to code those same datasets that ranged from 95 percent to 100 percent. In its current form, we find that NIOCCS is a tool that might be best used to reduce the number of cases human coders must code, either in coding historical data to a consistent codeframe or in coding data from future HRS waves. However, it is not yet ready to fully replace human coders.

Pages: 22 pages
Date: 2018-12
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
https://mrdrc.isr.umich.edu/publications/papers/pdf/wp392.pdf (application/pdf)
Our link check indicates that this URL is bad, the error code is: 403 Forbidden

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:mrr:papers:wp392

Access Statistics for this paper

More papers in Working Papers from University of Michigan, Michigan Retirement Research Center P.O. Box 1248, Ann Arbor, MI 48104. Contact information at EDIRC.
Bibliographic data for series maintained by MRRC Administrator ().