A Quantitative Method for Substantive Robustness Assessment

Justin Esarey and Nathan Danneman

Political Science Research and Methods, 2015, vol. 3, issue 1, 95-111

Abstract: Empirical political science is not simply about reporting evidence; it is also about coming to conclusions on the basis of that evidence and acting on those conclusions. But whether a result is substantively significant––strong and certain enough to justify acting upon the belief that the null hypothesis is false––is difficult to objectively pin down, in part because different researchers have different standards for interpreting evidence. Instead, this article advocates judging results according to their “substantive robustness,” the degree to which a community with heterogeneous standards for interpreting evidence would agree that the result is substantively significant. This study illustrates how this can be done using Bayesian statistical decision techniques. Judging results in this way yields a tangible benefit: false positives are reduced without decreasing the power of the test, thus decreasing the error rate in published results.

Date: 2015
References: Add references at CitEc
Citations: View citations in EconPapers (1)

Downloads: (external link)
https://www.cambridge.org/core/product/identifier/ ... type/journal_article link to article abstract page (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:cup:pscirm:v:3:y:2015:i:01:p:95-111_00

Access Statistics for this article

More articles in Political Science Research and Methods from Cambridge University Press Cambridge University Press, UPH, Shaftesbury Road, Cambridge CB2 8BS UK.
Bibliographic data for series maintained by Kirk Stebbing ().