Measuring the Unmeasurable? Systematic Evidence on Scale Transformations in Subjective Survey Data

Caspar F. Kaiser () and Anthony Lepinteur ()
Additional contact information
Caspar F. Kaiser: Warwick Business School
Anthony Lepinteur: University of Luxembourg

No 18029, IZA Discussion Papers from Institute of Labor Economics (IZA)

Abstract: Economists routinely use survey measures of, for example, risk preferences, trust, political attitudes, or wellbeing. The literature generally treats numerical response categories as if they represent equal psychological intervals. We provide the first systematic test of this assumption, developing a general framework to quantify how easily results can be overturned when this linearity assumption is relaxed. Using original experimental data, we show that respondents interpret survey scales in ways that do deviate from linearity, but only mildly. Focusing on wellbeing research, we then replicate 30,000+ coefficient estimates across more than 80 papers published in top economics journals. Replicated coefficient signs are remarkably robust to mild departures from linear scale-use. However, statistical inference and estimates of relative effect magnitudes become unreliable, even under modest departures from linearity. This is especially problematic for policy applications. We show that these concerns generalise to many other widely used survey-based constructs.

Keywords: life satisfaction; wellbeing; ordinal scales; Likert scales; survey methods (search for similar items in EconPapers)
JEL-codes: C18 C87 I31 (search for similar items in EconPapers)
Date: 2025-07
References: Add references at CitEc
Citations:

Downloads: (external link)
https://docs.iza.org/dp18029.pdf (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:iza:izadps:dp18029

Ordering information: This working paper can be ordered from
IZA, Margard Ody, P.O. Box 7240, D-53072 Bonn, Germany

Access Statistics for this paper

More papers in IZA Discussion Papers from Institute of Labor Economics (IZA) IZA, P.O. Box 7240, D-53072 Bonn, Germany. Contact information at EDIRC.
Bibliographic data for series maintained by Holger Hinte ().