A User-Driven Method for Using Research Products to Empirically Assess Item Importance in National Surveys

Ong Ai Rene (), Schultz Robert (), Sinozich Sofi (), Sinibaldi Jennifer (), West Brady T (), Wagner James and Finamore John ()
Additional contact information
Ong Ai Rene: Institute for Social Research, University of Michigan, 426 Thompson St, Ann Arbor, MI 48104 U.S.A.
Schultz Robert: Institute for Social Research, University of Michigan, 426 Thompson St, Ann Arbor, MI 48104 U.S.A.
Sinozich Sofi: University of Maryland, College Park, MD 20742, U.S.A.
Sinibaldi Jennifer: The National Center for Science and Engineering Statistics (NCSES) within the National Science Foundation, 2415 Eisenhower Avenue, Alexandria, Virginia 22314, U.S.A.
West Brady T: Institute for Social Research, University of Michigan, 426 Thompson St, Ann Arbor, MI 48104 U.S.A.
Wagner James: Institute for Social Research, University of Michigan, 426 Thompson St, Ann Arbor, MI 48104 U.S.A.
Finamore John: The National Center for Science and Engineering Statistics (NCSES) within the National Science Foundation, 2415 Eisenhower Avenue, Alexandria, Virginia 22314, U.S.A.

Journal of Official Statistics, 2022, vol. 38, issue 4, 1235-1251

Abstract: Large-scale, nationally representative surveys serve many vital functions, but these surveys can be long and burdensome for respondents. Cutting survey length can help to reduce respondent burden and may improve data quality but removing items from these surveys is not a trivial matter. We propose a method to empirically assess item importance and associated burden in national surveys and guide this decision-making process using different research products produced from such surveys. This method is demonstrated using the Survey of Doctorate Recipients (SDR), a biennial survey administered to individuals with a science, engineering, and health doctorate. We used three main sources of information on the SDR variables: a bibliography of documents using the SDR data as a measure of item use and importance, SDR data table download statistics from the Scientists and Engineers Statistical Data System as an additional measure of item use, and web timing paradata and break-off rates as a measure of burden. Putting this information together, we identified 35 unused items (17% of the survey) and found that the most burdensome items are highly important. We conclude with general recommendations for those hoping to employ similar methodologies in the future.

Keywords: Respondent burden; public use data; survey length (search for similar items in EconPapers)
Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://doi.org/10.2478/jos-2022-0052 (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:vrs:offsta:v:38:y:2022:i:4:p:1235-1251:n:1

DOI: 10.2478/jos-2022-0052

Access Statistics for this article

Journal of Official Statistics is currently edited by Annica Isaksson and Ingegerd Jansson

More articles in Journal of Official Statistics from Sciendo
Bibliographic data for series maintained by Peter Golla ().