WP 76 - Comparing different weighting procedures for volunteer web surveys
Stephanie Steinmetz (),
Kea Gartje Tijdens and
Pablo Pedraza García ()
Additional contact information Stephanie Steinmetz: Faculteit der Maatschappij- en Gedragswetenschappen – Afdeling Sociologie en Antropologie, Universiteit van Amsterdam, http://home.medewerker.uva.nl/s.m.steinmetz Pablo Pedraza García: University of Salamanca
The strengths and weaknesses of web surveys have been widely described in the literature. Of particular interest is the question to which degree the obtained results can be generalised for the whole population? To deal with this problem weighting adjustments, like post-stratification and propensity score adjustment (PSA) have been seen as a possible solution. In the scientific community, however, particularly PSA has traditionally not been applied in the field of surveys, and there has been a minimal amount of evidence for its applicability and performance, and the implications are not conclusive. Against this background, the paper attempts to explore the two statistical weighting procedures for the German and Dutch WageIndicator Survey 2006. To evaluate the effectiveness of the weighting techniques in adjusting biases arising from non-randomised sample selection, the existing selection bias has been explored and the efficiency of the weights has be tested by comparing un-weighted and weighted results with those that could be found using data from the German SOEP and the Dutch OSA Panel for the same year. The results reveal that the impact of the applied weights is very limited and that the different weighting methods using balancing variables do not make web survey data more comparable to the general population. This holds for the German as well as for the Dutch sample.