The edit distance approach: an alternate method for assessing multi-observer agreement in process studies

Rita Snyder, José Vidal, Bo Cai and Nathan Huynh

Health Systems, 2016, vol. 5, issue 1, 6-12

Abstract: Direct observation of complex health-care processes typically involves multi-observer recording of sequential process tasks. Inference, the key validity threat to multi-observer recording, is controlled with observer training and assessment for the degree of recording consistency across observers. The gold standard for assessing recording consistency is the Kappa statistic, which assumes an exact task sequence match among observers. This assumption, however, is often difficult to meet with health-care process observations where task speed and complexity can result in uneven task sequence recording among observers. The edit distance approach, derived from information string theory, is not predicated on an exact task sequence match and offers an alternative to the Kappa statistic for assessing multi-observer agreement. The paper uses simultaneously recorded process observations with uneven task sequences made by three observers to compare agreement results for the edit distance approach and Kappa statistic. Edit distance approach strengths and limitations are discussed.

Date: 2016
References: Add references at CitEc
Citations:

Downloads: (external link)
http://hdl.handle.net/10.1057/hs.2014.32 (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:taf:thssxx:v:5:y:2016:i:1:p:6-12

Ordering information: This journal article can be ordered from
http://www.tandfonline.com/pricing/journal/thss20

DOI: 10.1057/hs.2014.32

Access Statistics for this article

Health Systems is currently edited by Sally Brailsford

More articles in Health Systems from Taylor & Francis Journals
Bibliographic data for series maintained by Chris Longhurst ().