A cautionary note on the reliability of the online survey data - the case of Wage Indicator
Joanna Tyrowicz (),
Magdalena Smyk () and
Lucas van der Velde
No 26, GRAPE Working Papers from GRAPE Group for Research in Applied Economics
We investigate the reliability of data from the Wage Indicator (WI), the largest online survey on earnings and working conditions. Comparing WI to nationally representative data sources for 17 countries reveals that participants of WI are not likely to have been representatively drawn from the respective populations. Previous literature has proposed to utilize weights based on inverse propensity scores, but this procedure was shown to leave reweighted WI samples different from the benchmark nationally representative data. We propose a novel procedure, building on covariate balancing propensity score, which achieves complete reweighting of the WI data, making it able to replicate the structure of nationally representative samples on observable characteristics. While rebalancing assures the match between WI and representative benchmark data sources, we show that the wage schedules remain different for a large group of countries. Using the example of a Mincerian wage regression, we find that in more than a third of the cases, our proposed novel reweighting assures that estimates obtained on WI data are not biased relative to nationally representative data. However, in the remaining 60% of the analyzed 95 datasets systematic differences in the estimated coefficients of the Mincerian wage regression between WI and nationally representative data persists even after reweighting. We provide some intuition about the reasons behind these biases. Notably, objective factors such as access to the Internet or richness appear to matter, but self-selection (on unobservable characteristics) among WI participants appears to constitute an important source of bias.
Keywords: Wage Indicator; online surveys; propensity score matching; weights (search for similar items in EconPapers)
JEL-codes: C81 J30 J31 (search for similar items in EconPapers)
New Economics Papers: this item is included in nep-age and nep-dge
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (2) Track citations by RSS feed
Downloads: (external link)
Working Paper: A cautionary note on the reliability of the online survey data – the case of Wage Indicator (2018)
Working Paper: A Cautionary Note on the Reliability of the Online Survey Data: The Case of Wage Indicator (2018)
This item may be available elsewhere in EconPapers: Search for items with the same title.
Export reference: BibTeX
RIS (EndNote, ProCite, RefMan)
Persistent link: https://EconPapers.repec.org/RePEc:fme:wpaper:26
Access Statistics for this paper
More papers in GRAPE Working Papers from GRAPE Group for Research in Applied Economics Contact information at EDIRC.
Bibliographic data for series maintained by Jan Hagemejer ().