Are 'Webographic' or Attitudinal Questions Useful for Adjusting Estimates From Web Surveys Using Propensity Scoring?

Matthias Schonlau, Arthur van Soest () and Arie Kapteyn

No WR-506, Working Papers from RAND Corporation

Abstract: Inference from Web surveys may be affected by non-random selection of Web survey participants. One approach to reduce selection bias is to use propensity scores and a parallel phone survey. This approach uses demographic and additional so-called Webographic or lifestyle variables to balance observed differences between Web survey respondents and phone survey respondents. Here the authors investigate some of the Webographic questions used by Harris Interactive, a commercial company specializing in Web surveys. Their Webographic questions include choice of activities such as reading, sports and traveling and perceptions about what would constitute a violation of privacy. They use data from an existing probability sample of respondents over 40 who are interviewed over the phone, and a corresponding sample of respondents interviewed over the Web. They find that Webographic questions differentiate between on and offline populations differently than demographic questions. In general, propensity score adjustment of variables in the Web survey works quite well for a number of variables of interest (including home ownership and labor force participation). For two outcomes, (having emotional problems and often experiencing pain) the process of adjusting for demographic variables leads to the discovery of an instance of Simpson’s paradox, implying a differential mode effect or differential selection. They interpret this mainly as the result of a mode effect, where sensitive questions are more likely to receive a positive response over the Internet than over the phone.

Keywords: propensity scoring; Web survey; selection bias; Webographic (search for similar items in EconPapers)
JEL-codes: C42 C80 (search for similar items in EconPapers)
Pages: 18 pages
Date: 2007-07
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (7)

Downloads: (external link)
https://www.rand.org/content/dam/rand/pubs/working_papers/2007/RAND_WR506.pdf (application/pdf)
Our link check indicates that this URL is bad, the error code is: 403 Forbidden

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:ran:wpaper:wr-506

Access Statistics for this paper

More papers in Working Papers from RAND Corporation Contact information at EDIRC.
Bibliographic data for series maintained by Benson Wong ().