Sensitivity of self-reported noncognitive skills to survey administration conditions

Yuanyuan Chen, Shuaizhang Feng, James Heckman and Tim Kautz
Additional contact information
Yuanyuan Chen: Institute for Advanced Research, Key Laboratory of Mathematical Economics of Ministry of Education, Shanghai University of Finance and Economics, Shanghai 200433, China
Shuaizhang Feng: Institute for Economic and Social Research, Jinan University, Guangzhou 510632, China
Tim Kautz: Mathematica, Inc., Princeton, NJ 08540

Proceedings of the National Academy of Sciences, 2020, vol. 117, issue 2, 931-935

Abstract: Noncognitive skills (e.g., persistence and self-control) are typically measured using self-reported questionnaires in which respondents rate their own skills. In many applications—including program evaluation and school accountability systems—such reports are assumed to measure only the skill of interest. However, self-reports might also capture other dimensions aside from the skill, such as aspects of a respondent’s situation, which could include incentives and the conditions in which they complete the questionnaire. To explore this possibility, this study conducted 2 experiments to estimate the extent to which survey administration conditions can affect student responses on noncognitive skill questionnaires. The first experiment tested whether providing information about the importance of noncognitive skills to students directly affects their responses, and the second experiment tested whether incentives tied to performance on another task indirectly affect responses. Both experiments suggest that self-reports of noncognitive skills are sensitive to survey conditions. The effects of the conditions are relatively large compared with those found in the program evaluation literature, ranging from 0.05 to 0.11 SDs. These findings suggest that the effects of interventions or other social policies on self-reported noncognitive skills should be interpreted with caution.

Keywords: noncognitive skills; psychological assessment; personality traits; Big Five; incentives (search for similar items in EconPapers)
Date: 2020
References: Add references at CitEc
Citations: View citations in EconPapers (12)

Downloads: (external link)
http://www.pnas.org/content/117/2/931.full (application/pdf)

Related works:
Working Paper: Sensitivity of Self-Reported Noncognitive Skills to Survey Administration Conditions
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:nas:journl:v:117:y:2020:p:931-935

Access Statistics for this article

More articles in Proceedings of the National Academy of Sciences from Proceedings of the National Academy of Sciences
Bibliographic data for series maintained by PNAS Product Team ().