On the choice of measures of reliability and validity in the content-analysis of texts

Anton Oleinik (), Irina Popova, Svetlana Kirdina and Tatyana Shatalova
Authors registered in the RePEc Author Service: Светлана Георгиевна Кирдина-Чэндлер ()

Quality & Quantity: International Journal of Methodology, 2014, vol. 48, issue 5, 2703-2718

Abstract: The paper discusses several reliability measures: Scott’s pi, Krippendorff’s alpha, free marginal adjustment (Bennett, Alpert and Goldstein’s $$S$$ S ), Cohen’s kappa, and Perreault and Leigh’s $$I$$ I and the assumptions on which they are based. It is suggested that correlation coefficients between, on one hand, the distribution of qualitative codes and, on the other hand, word co-occurrences and the distribution of the categories identified with the help of the dictionary based on substitution complement the other reliability measures. The paper shows that the choice of the reliability measure depends on the format of the text (stylistic versus rhetorical) and the type of reading (comprehension versus interpretation). Namely, Cohen’s kappa and Bennett, Alpert and Goldstein’s $$S$$ S emerge as reliability measures particularly suited for perspectival reading of rhetorical texts. Outcomes of the content analysis of 57 texts performed by four coders with the help of computer program QDA Miner inform the analysis. Copyright Springer Science+Business Media Dordrecht 2014

Keywords: Reliability measures; Content analysis; Correlation analysis; Interpretation; Comprehension; Stylistic texts; Rhetorical texts (search for similar items in EconPapers)
Date: 2014
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (7)

Downloads: (external link)
http://hdl.handle.net/10.1007/s11135-013-9919-0 (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:spr:qualqt:v:48:y:2014:i:5:p:2703-2718

Ordering information: This journal article can be ordered from
http://www.springer.com/economics/journal/11135

DOI: 10.1007/s11135-013-9919-0

Access Statistics for this article

Quality & Quantity: International Journal of Methodology is currently edited by Vittorio Capecchi

More articles in Quality & Quantity: International Journal of Methodology from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().