A cautionary note on the reliability of the online survey data - the case of Wage Indicator

Joanna Tyrowicz, Magdalena Smyk () and Lucas van der Velde
Additional contact information
Magdalena Smyk: Group for Research in Applied Economics (GRAPE)

No 26, GRAPE Working Papers from GRAPE Group for Research in Applied Economics

Abstract: We investigate the reliability of data from the Wage Indicator (WI), the largest online survey on earnings and working conditions. Comparing WI to nationally representative data sources for 17 countries reveals that participants of WI are not likely to have been representatively drawn from the respective populations. Previous literature has proposed to utilize weights based on inverse propensity scores, but this procedure was shown to leave reweighted WI samples different from the benchmark nationally representative data. We propose a novel procedure, building on covariate balancing propensity score, which achieves complete reweighting of the WI data, making it able to replicate the structure of nationally representative samples on observable characteristics. While rebalancing assures the match between WI and representative benchmark data sources, we show that the wage schedules remain different for a large group of countries. Using the example of a Mincerian wage regression, we find that in more than a third of the cases, our proposed novel reweighting assures that estimates obtained on WI data are not biased relative to nationally representative data. However, in the remaining 60% of the analyzed 95 datasets systematic differences in the estimated coefficients of the Mincerian wage regression between WI and nationally representative data persists even after reweighting. We provide some intuition about the reasons behind these biases. Notably, objective factors such as access to the Internet or richness appear to matter, but self-selection (on unobservable characteristics) among WI participants appears to constitute an important source of bias.

Keywords: Wage Indicator; online surveys; propensity score matching; weights (search for similar items in EconPapers)
JEL-codes: C81 J30 J31 (search for similar items in EconPapers)
Pages: 36 pages
Date: 2018
New Economics Papers: this item is included in nep-age and nep-dge
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (7)

Downloads: (external link)
http://grape.org.pl/WP/26_SmykTyrowiczVanDerVelde_website.pdf (application/pdf)

Related works:
Journal Article: A Cautionary Note on the Reliability of the Online Survey Data: The Case of Wage Indicator (2021)
Working Paper: A cautionary note on the reliability of the online survey data – the case of Wage Indicator (2018)
Working Paper: A Cautionary Note on the Reliability of the Online Survey Data: The Case of Wage Indicator (2018)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:fme:wpaper:26

Access Statistics for this paper

More papers in GRAPE Working Papers from GRAPE Group for Research in Applied Economics Contact information at EDIRC.
Bibliographic data for series maintained by Jan Hagemejer ().