Validating an Index of Selection Bias for Proportions in Non‐Probability Samples

Angelina Hammon and Sabine Zinn

International Statistical Review, 2025, vol. 93, issue 3, 499-516

Abstract: Fast online surveys without sampling frames are becoming increasingly important in survey research. Their recruitment methods result in non‐probability samples. As the mechanism of data generation is always unknown in such samples, the problem of non‐ignorability arises making vgeneralisation of calculated statistics to the population of interest highly questionable. Sensitivity analyses provide a valuable tool to deal with non‐ignorability. They capture the impact of different sample selection mechanisms on target statistics. In 2019, Andridge and colleagues proposed an index to quantify potential (non‐ignorable) selection bias in proportions that combines the effects of different selection mechanisms. In this paper, we validate this index with an artificial non‐probability sample generated from a large empirical data set and additionally applied it to proportions estimated from data on current political attitudes arising from a real non‐probability sample selected via River sampling. We find a number of conditions that must be met for the index to perform meaningfully. When these requirements are fulfilled, the index shows an overall good performance in both of our applications in detecting and correcting present selection bias in estimated proportions. Thus, it provides a powerful measure for evaluating the robustness of results obtained from non‐probability samples.

Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
https://doi.org/10.1111/insr.12590

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bla:istatr:v:93:y:2025:i:3:p:499-516

Ordering information: This journal article can be ordered from
http://www.blackwell ... bs.asp?ref=0306-7734

Access Statistics for this article

International Statistical Review is currently edited by Eugene Seneta and Kees Zeelenberg

More articles in International Statistical Review from International Statistical Institute Contact information at EDIRC.
Bibliographic data for series maintained by Wiley Content Delivery ().