When is a Match Sufficient? A Score-based Balance Metric for the Synthetic Control Method

Parast Layla (), Hunt Priscillia, Griffin Beth Ann and David Powell
Additional contact information
Parast Layla: RAND Corporation, 1776Main Street, Santa Monica, CAU.S.A
Hunt Priscillia: RAND Corporation, 1776Main Street, Santa Monica, CAU.S.A
Griffin Beth Ann: RAND Corporation, 1200 S Hayes St, Arlington, VA22202U.S.A

Journal of Causal Inference, 2020, vol. 8, issue 1, 209-228

Abstract: In some applications, researchers using the synthetic control method (SCM) to evaluate the effect of a policy may struggle to determine whether they have identified a “good match” between the control group and treated group. In this paper, we demonstrate the utility of the mean and maximum Absolute Standardized Mean Difference (ASMD) as a test of balance between a synthetic control unit and treated unit, and provide guidance on what constitutes a poor fit when using a synthetic control. We explore and compare other potential metrics using a simulation study. We provide an application of our proposed balance metric to the 2013 Los Angeles (LA) Firearm Study [9]. Using Uniform Crime Report data, we apply the SCM to obtain a counterfactual for the LA firearm-related crime rate based on a weighted combination of control units in a donor pool of cities. We use this counterfactual to estimate the effect of the LA Firearm Study intervention and explore the impact of changing the donor pool and pre-intervention duration period on resulting matches and estimated effects. We demonstrate how decision-making about the quality of a synthetic control can be improved by using ASMD. The mean and max ASMD clearly differentiate between poor matches and good matches. Researchers need better guidance on what is a meaningful imbalance between synthetic control and treated groups. In addition to the use of gap plots, the proposed balance metric can provide an objective way of determining fit.

Keywords: Synthetic control method; Non-experimental study; Matching methods; Gun violence (search for similar items in EconPapers)
Date: 2020
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://doi.org/10.1515/jci-2020-0013 (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bpj:causin:v:8:y:2020:i:1:p:209-228:n:10

DOI: 10.1515/jci-2020-0013

Access Statistics for this article

Journal of Causal Inference is currently edited by Elias Bareinboim, Jin Tian and Iván Díaz

More articles in Journal of Causal Inference from De Gruyter
Bibliographic data for series maintained by Peter Golla ().