What Fuels Publication Bias?: Theoretical and Empirical Analyses of Risk Factors Using the Caliper Test

Auspurg Katrin () and Hinz Thomas ()
Additional contact information
Auspurg Katrin: Department of History and Sociology, Box D40, University of Konstanz, 78457 Konstanz, Germany
Hinz Thomas: Department of History and Sociology, Box D40, University of Konstanz, 78457 Konstanz, Germany

Journal of Economics and Statistics (Jahrbuecher fuer Nationaloekonomie und Statistik), 2011, vol. 231, issue 5-6, 636-660

Abstract: Significance tests were originally developed to enable more objective evaluations of research results. Yet the strong orientation towards statistical significance encourages biased results, a phenomenon termed “publication bias”. Publication bias occurs whenever the likelihood or time-lag of publication, or the prominence, language, impact factor of journal space or the citation rate of studies depend on the direction and significance of research findings.Although there is much evidence concerning the existence of publication bias in all scientific disciplines and although its detrimental consequences for the progress of the sciences have been known for a long time, all attempts to eliminate the bias have failed. The present article reviews the history and logic of significance testing, the state of research on publication bias, and existing practical recommendations. After demonstrating that more systematical research on the risk factors of publication bias is needed, the paper suggests two new directions for publication bias research. First, a more comprehensive theoretical model based on theories of rational choice and economics as well as on the sociology of science is sketched out. Publication bias is recognized as the outcome of a social dilemma that cannot be overcome by moral pleas alone. Second, detection methods for publication bias going beyond meta-analysis, ones that are more suitable for testing causal hypotheses, are discussed. In particular, the “caliper test” seems well-suited for conducting theoretically motivated comparisons across heterogeneous research fields like sociology. Its potential is demonstrated by testing hypotheses on (a) the relevance of explicitly vs. implicitly stated research propositions and on (b) the relevance of the number of authors on incidence rates of publication bias in 50 papers published in leading German sociology journals.

Keywords: Significance testing; publication bias; caliper test; rational choice; sociology of science; Significance testing; publication bias; caliper test; rational choice; sociology of science (search for similar items in EconPapers)
Date: 2011
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (8)

Downloads: (external link)
https://doi.org/10.1515/jbnst-2011-5-607 (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:jns:jbstat:v:231:y:2011:i:5-6:p:636-660

DOI: 10.1515/jbnst-2011-5-607

Access Statistics for this article

Journal of Economics and Statistics (Jahrbuecher fuer Nationaloekonomie und Statistik) is currently edited by Peter Winker

More articles in Journal of Economics and Statistics (Jahrbuecher fuer Nationaloekonomie und Statistik) from De Gruyter
Bibliographic data for series maintained by Peter Golla ().