How and how much does expert error matter? Implications for quantitative peace research

Kyle L Marquardt
Additional contact information
Kyle L Marquardt: School of Politics and Governance & International Center for the Study of Institutions and Development, 3570National Research University Higher School of Economics

Journal of Peace Research, 2020, vol. 57, issue 6, 692-700

Abstract: Expert-coded datasets provide scholars with otherwise unavailable data on important concepts. However, expert coders vary in their reliability and scale perception, potentially resulting in substantial measurement error. These concerns are acute in expert coding of key concepts for peace research. Here I examine (1) the implications of these concerns for applied statistical analyses, and (2) the degree to which different modeling strategies ameliorate them. Specifically, I simulate expert-coded country-year data with different forms of error and then regress civil conflict onset on these data, using five different modeling strategies. Three of these strategies involve regressing conflict onset on point estimate aggregations of the simulated data: the mean and median over expert codings, and the posterior median from a latent variable model. The remaining two strategies incorporate measurement error from the latent variable model into the regression process by using multiple imputation and a structural equation model. Analyses indicate that expert-coded data are relatively robust: across simulations, almost all modeling strategies yield regression results roughly in line with the assumed true relationship between the expert-coded concept and outcome. However, the introduction of measurement error to expert-coded data generally results in attenuation of the estimated relationship between the concept and conflict onset. The level of attenuation varies across modeling strategies: a structural equation model is the most consistently robust estimation technique, while the median over expert codings and multiple imputation are the least robust.

Keywords: Bayesian methods; civil conflict; conflict onset; ethnic politics; expert-coded data; latent variable models (search for similar items in EconPapers)
Date: 2020
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (4)

Downloads: (external link)
https://journals.sagepub.com/doi/10.1177/0022343320959121 (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:sae:joupea:v:57:y:2020:i:6:p:692-700

DOI: 10.1177/0022343320959121

Access Statistics for this article

More articles in Journal of Peace Research from Peace Research Institute Oslo
Bibliographic data for series maintained by SAGE Publications ().