Reliability procedures for categorical data in Performance Analysis

Nic James, Joe Taylor and Simon Stanley

International Journal of Performance Analysis in Sport, 2007, vol. 7, issue 1, 1-11

Abstract: A number of papers have discussed reliability assessment techniques but none have previously limited themselves to methods appropriate to the collection of categorical data on the nominal scale i.e. where events are recorded within defined categories, as in many instances for performance analysis. This paper makes a number of recommendations including the use of a gold standard coding to compare the typical analyst’s codes against. Reliability measures should reflect the way in which the notation data is analysed so that assessments can be made regarding the extent to which each variable presented in the results was coded accurately. The Yule’s Q statistic, based on the odds ratio, is recommended as the reliability statistic of choice for categorical data due to the intuitiveness of the measure, its ease of calculation and its apparent applicability to the decision making of performance analysts. Furthermore a value of 0.95 or above should be deemed an acceptable value to determine confidence in the analyst’s ability to place an event into a category reliably.

Date: 2007
References: View complete reference list from CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
http://hdl.handle.net/10.1080/24748668.2007.11868382 (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:taf:rpanxx:v:7:y:2007:i:1:p:1-11

Ordering information: This journal article can be ordered from
http://www.tandfonline.com/pricing/journal/RPAN20

DOI: 10.1080/24748668.2007.11868382

Access Statistics for this article

International Journal of Performance Analysis in Sport is currently edited by Peter O'Donoghue

More articles in International Journal of Performance Analysis in Sport from Taylor & Francis Journals
Bibliographic data for series maintained by Chris Longhurst ().