Measuring the Unmeasurable? Systematic Evidence on Scale Transformations in Subjective Survey Data

Caspar Kaiser and Anthony Lepinteur

No 3gmzy_v1, SocArXiv from Center for Open Science

Abstract: Economists routinely use survey measures of, for example, risk preferences, trust, political attitudes, or wellbeing. The literature generally treats numerical response categories as if they represent equal psychological intervals. We provide the first systematic test of this assumption, developing a general framework to quantify how easily results can be overturned when this linearity assumption is relaxed. Using original experimental data, we show that respondents interpret survey scales in ways that do deviate from linearity, but only mildly. Focusing on wellbeing research, we then replicate 30,000+ coefficient estimates across more than 80 papers published in top economics journals. Replicated coefficient signs are remarkably robust to mild departures from linear scale-use. However, statistical inference and estimates of relative effect magnitudes become unreliable, even under modest departures from linearity. This is especially problematic for policy applications. We show that these concerns generalise to many other widely used survey-based constructs.

Date: 2025-07-23
New Economics Papers: this item is included in nep-hap
References: Add references at CitEc
Citations:

Downloads: (external link)
https://osf.io/download/687f6810ae4889aca21a1780/

Related works:
Working Paper: Measuring the Unmeasurable? Systematic Evidence on Scale Transformations in Subjective Survey Data (2025)
Working Paper: Measuring the Unmeasurable? Systematic Evidence on Scale Transformations in Subjective Survey Data (2025)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:osf:socarx:3gmzy_v1

DOI: 10.31219/osf.io/3gmzy_v1

Access Statistics for this paper

More papers in SocArXiv from Center for Open Science
Bibliographic data for series maintained by OSF ().