Simulated power analyses for observational studies: An application to the Affordable Care Act Medicaid expansion

Bernard Black, Alex Hollingsworth (), Letícia Nunes and Kosali Simon

Journal of Public Economics, 2022, vol. 213, issue C

Abstract: Power is an important factor in assessing the likely validity of a statistical estimate. An analysis with low power is unlikely to produce convincing evidence of a treatment effect even when one exists. Of greater concern, a statistically significant estimate from a low-powered analysis is likely to misstate the true effect size, including finding estimates of the wrong sign or that are several times too large. Yet statistical power is rarely reported in published economics work. This is in part because many modern research designs are complex enough that power cannot be easily ascertained using simple formulae. Power can also be difficult to estimate in observational settings. Using an applied example–the link between gaining health insurance and mortality–we conduct a simulated power analysis to outline the importance of power and ways to estimate power in complex research settings. We find that standard difference-in-differences and triple differences analyses of Medicaid expansions using county or state mortality data would need to induce reductions in population mortality of at least 2% to be well powered. While there is no single, correct method for conducting a simulated power analysis, our manuscript outlines how applied researchers can conduct simulations appropriate to their settings.

Keywords: Simulated power analysis; Health insurance; Mortality; Medicaid expansion (search for similar items in EconPapers)
JEL-codes: C15 I13 (search for similar items in EconPapers)
Date: 2022
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (7)

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S0047272722001153
Full text for ScienceDirect subscribers only

Related works:
Working Paper: Simulated Power Analyses for Observational Studies: An Application to the Affordable Care Act Medicaid Expansion (2019)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:pubeco:v:213:y:2022:i:c:s0047272722001153

DOI: 10.1016/j.jpubeco.2022.104713

Access Statistics for this article

Journal of Public Economics is currently edited by R. Boadway and J. Poterba

More articles in Journal of Public Economics from Elsevier
Bibliographic data for series maintained by Catherine Liu ().