Assessing the robustness of meta-analytic results in information systems: publication bias and outliers

Sven Kepes, Manoj A. Thomas, Frantz Rowe and Mark Silver

European Journal of Information Systems, 2018, vol. 27, issue 1, 90-123

Abstract: Meta-analytic studies serve to generate cumulative knowledge and guide evidence-based practice. However, publication bias and outliers threaten the accuracy and robustness of meta-analytic results. Unfortunately, most meta-analytic studies in information systems (IS) research do not assess the presence of these phenomena. Furthermore, some methods commonly used for the detection of publication bias are now recognised as inappropriate. We conduct a comprehensive assessment of four previously published meta-analytic studies in IS. We use multiple methods to assess the effects of publication bias and outliers on the meta-analytic results. Our findings indicate that publication bias and/or outliers have affected the results of three of the four meta-analytic studies. Some methods indicate that select meta-analytic means were misestimated by potentially more than 100%. Our analyses offer methodological exemplars that can be followed to assess the potential adverse effects of publication bias and outliers on meta-analytic results, including their combined effects. We make additional contributions to scientific knowledge by evaluating the performance of different publication bias assessment methods used across scientific disciplines. In brief, we highlight the importance of a rigorous assessment of publication bias and outliers on meta-analytic results to improve the trustworthiness of our cumulative knowledge and evidence-based practice.

Date: 2018
References: Add references at CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
http://hdl.handle.net/10.1080/0960085X.2017.1390188 (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:taf:tjisxx:v:27:y:2018:i:1:p:90-123

Ordering information: This journal article can be ordered from
http://www.tandfonline.com/pricing/journal/tjis20

DOI: 10.1080/0960085X.2017.1390188

Access Statistics for this article

European Journal of Information Systems is currently edited by Par Agerfalk

More articles in European Journal of Information Systems from Taylor & Francis Journals
Bibliographic data for series maintained by Chris Longhurst ().