Instability of estimation results based on caliper matching with propensity scores

Kazushi Maruo, Yusuke Yamaguchi, Ryota Ishii and Masahiko Gosho

PLOS ONE, 2025, vol. 20, issue 6, 1-14

Abstract: Caliper matching is often used to adjust for confounding biases in observational studies. This method with random order matching allows for the cherry-picking of the analysis results to suit the analyst’s convenience. Random order matching can also result in large fluctuations in the analysis results due to small additions and/or changes in data. These “instability problems” might compromise the reproducibility of the study results. Some studies have discussed instability issues, but the conditions are limited, and there is no knowledge of which alternative order method should be used instead of the random order method. We evaluate the instability problem by calculating the extent to which the results can vary within a single study dataset and provide guidelines for choosing the best alternative matching order method based on simulations and a case study. From simulation studies, instability might be serious when the sample size was small, the true odds ratio was large, the proportion for the treatment group was large, and the c-statistic for the propensity score model was large. We recommend not using random order matching and instead using lowest to highest score order matching or the median of multiple random order matching results. We also recommend pre-specifying the matching order method.

Date: 2025
References: View complete reference list from CitEc
Citations:

Downloads: (external link)
https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0325317 (text/html)
https://journals.plos.org/plosone/article/file?id= ... 25317&type=printable (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:plo:pone00:0325317

DOI: 10.1371/journal.pone.0325317

Access Statistics for this article

More articles in PLOS ONE from Public Library of Science
Bibliographic data for series maintained by plosone ().