Events Data as Bismarck's Sausages? Intercoder Reliability, Coders' Selection, and Data Quality

Andrea Ruggeri, Theodora-Ismene Gizelis and Han Dorussen

International Interactions, 2011, vol. 37, issue 3, 340-361

Abstract: Precise measurement is difficult but essential in the generation of high-quality data, and it is therefore remarkable that often so little attention is paid to intercoder reliability. It is commonly recognized that poor validity leads to systematic errors and biased inference. In contrast, low reliability is generally assumed to be a lesser concern, leading only to random errors and inefficiency. We evaluate the intercoder reliability of our recently collected data on governance events in UN peacekeeping and show how poor coding and low intercoder reliability can produce systematic errors and even biased inference. We also show how intercoder reliability checks are useful to improve data quality. Continuous testing for intercoder reliability ex post enables researchers to create better data and ultimately improves the quality of their analyses.

Date: 2011
References: View complete reference list from CitEc
Citations: View citations in EconPapers (5)

Downloads: (external link)
http://hdl.handle.net/10.1080/03050629.2011.596028 (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:taf:ginixx:v:37:y:2011:i:3:p:340-361

Ordering information: This journal article can be ordered from
http://www.tandfonline.com/pricing/journal/GINI20

DOI: 10.1080/03050629.2011.596028

Access Statistics for this article

International Interactions is currently edited by Michael Colaresi and Gerald Schneider

More articles in International Interactions from Taylor & Francis Journals
Bibliographic data for series maintained by Chris Longhurst ().