Meta-dominance analysis - A tool for the assessment of the quality of digital behavioural data

Andreas Schneck and Wojtek Przepiorka
Additional contact information
Wojtek Przepiorka: Utrecht University

No cy3wj, SocArXiv from Center for Open Science

Abstract: We propose a simple yet comprehensive conceptual framework for the identification of different sources of error in research with digital behavioral data. We use our framework to map potential sources of error in 25 years of research on reputation effects in peer-to-peer online market platforms. Using a meta-dataset comprising 346 effect sizes extracted from 109 articles, we apply meta-dominance analysis to quantify the relative importance of different error components. Our results indicate that 85% of explained effect size heterogeneity can be attributed to the measurement process, which comprises the choice of platform, data collection mode, construct operationalisation and variable transformation. Error components attributable to the sampling process or publication bias capture relatively small parts of the explained effect size heterogeneity. This approach reveals at which stages of the research process researcher decisions may affect data quality most. This approach can be used to identify potential sources of error in established strands of research beyond the literature of behavioral data from online platforms.

Date: 2023-03-11
New Economics Papers: this item is included in nep-pay
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://osf.io/download/640b992f86049007ac08681b/

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:osf:socarx:cy3wj

DOI: 10.31219/osf.io/cy3wj

Access Statistics for this paper

More papers in SocArXiv from Center for Open Science
Bibliographic data for series maintained by OSF ().