Evaluating Different Covariate Balancing Methods: A Monte Carlo Simulation

Fukui Hideki ()
Additional contact information
Fukui Hideki: Faculty of Law and Letters, Ehime University, 3 Bunkyo-Cho, Matsuyama, Ehime 790-8577, Japan

Statistics, Politics and Policy, 2023, vol. 14, issue 2, 205-326

Abstract: We investigate the effectiveness of five weighting and matching techniques, including propensity score matching (PSM), in improving covariate balance and reducing bias when estimating treatment effects in finite-sample situations through Monte Carlo simulations. King and Nielsen (2019. “Why Propensity Scores Should Not Be Used for Matching.” Political Analysis 27 (4): 435–54) argue that pruning observations based on PSM with 1-to-1 greedy matching can worsen rather than improve covariate balance and can increase the bias in the estimates of treatment effects. In our simulations, we observed this phenomenon not only in PSM with 1-to-1 greedy matching but also in other covariate balancing techniques that King and Nielsen (2019. “Why Propensity Scores Should Not Be Used for Matching.” Political Analysis 27 (4): 435–54) recommend as better matching methods, i.e. Mahalanobis distance matching (MDM) and coarsened exact matching (CEM). Regardless of the weighting/matching techniques and the data generation processes in this study, our findings indicate that matching and weighting under extreme caliper or cut-point settings does not improve covariate balance. In addition, once a substantially improved covariate balance is achieved in a given sample, the estimated bias tends to worsen slightly as the covariate balance continues to improve. Moreover, our simulation results suggest that OLS with proper covariates reduces selection bias as well as other weighting and matching methods. The results suggest that when analyzing observational data, it is important to avoid looking for a one-size-fits-all estimator and to identify the appropriate nonexperimental estimator carefully for the sample by thoroughly investigating the available data’s characteristics.

Keywords: bias correction; covariate balancing; Monte Carlo simulation; observed/unobserved confounders (search for similar items in EconPapers)
Date: 2023
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://doi.org/10.1515/spp-2022-0019 (text/html)
For access to full text, subscription to the journal or payment for the individual article is required.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bpj:statpp:v:14:y:2023:i:2:p:205-326:n:6

Ordering information: This journal article can be ordered from
https://www.degruyter.com/journal/key/spp/html

DOI: 10.1515/spp-2022-0019

Access Statistics for this article

Statistics, Politics and Policy is currently edited by Joel A. Middleton

More articles in Statistics, Politics and Policy from De Gruyter
Bibliographic data for series maintained by Peter Golla ().