The Role of Sample Size to Attain Statistically Comparable Groups – A Required Data Preprocessing Step to Estimate Causal Effects With Observational Data

Ana Kolar and Peter M. Steiner

Evaluation Review, 2021, vol. 45, issue 5, 195-227

Abstract: Background: Propensity score methods provide data preprocessing tools to remove selection bias and attain statistically comparable groups - the first requirement when attempting to estimate causal effects with observational data. Although guidelines exist on how to remove selection bias when groups in comparison are large, not much is known on how to proceed when one of the groups in comparison, for example, a treated group, is particularly small, or when the study also includes lots of observed covariates (relative to the treated group's sample size). Objectives: This article investigates whether propensity score methods can help us to remove selection bias in studies with small treated groups and large amount of observed covariates. Measures: We perform a series of simulation studies to study factors such as sample size ratio of control to treated units, number of observed covariates and initial imbalances in observed covariates between the groups of units in comparison, that is, selection bias. Results: The results demonstrate that selection bias can be removed with small treated samples, but under different conditions than in studies with large treated samples. For example, a study design with 10 observed covariates and eight treated units will require the control group to be at least 10 times larger than the treated group, whereas a study with 500 treated units will require at least, only, two times bigger control group. Conclusions: To confirm the usefulness of simulation study results for practice, we carry out an empirical evaluation with real data. The study provides insights for practice and directions for future research.

Keywords: causal inference; bias removal; propensity score methods; matching; experimental and observational study designs (search for similar items in EconPapers)
Date: 2021
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://journals.sagepub.com/doi/10.1177/0193841X211053937 (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:sae:evarev:v:45:y:2021:i:5:p:195-227

DOI: 10.1177/0193841X211053937

Access Statistics for this article

More articles in Evaluation Review
Bibliographic data for series maintained by SAGE Publications ().