Ziliak and McCloskey's Criticisms of Significance Tests: An Assessment

Thomas Mayer

Econ Journal Watch, 2012, vol. 9, issue 3, 256-297

Abstract: Stephen Ziliak and D. N. McCloskey have sharply criticized the prevailing use of significance tests. Their work has, in turn, come under vigorous attack. The vehemence of the debate may induce readers to wrongly dismiss it as a “he said-she said” debate, or else to take sides in an unbending way that does not do justice to valid points raised by the other side. This paper aims at a more balanced reading. While Ziliak and McCloskey claim that a substantial majority of economists who use significance tests confuse statistical with substantive significance, or commit the logical error of the transposed conditional, I argue that such errors are much less frequent than they claim, though still much too pervasive. They also argue that since significance tests focus on the existence of an effect rather than on its size, the tests do not answer scientific questions. I respond with counter-examples. Ziliak and McCloskey also complain that significance tests ignore loss functions. I argue that loss functions should be introduced only at a later stage. Ziliak and McCloskey are correct, however, that confidence intervals deserve much more emphasis. The most valuable message of their work is that significance tests should be treated less mechanically.

Keywords: Significance tests; t’s; p’s; confidence intervals; Ziliak; McCloskey; oomph (search for similar items in EconPapers)
JEL-codes: B4 C12 (search for similar items in EconPapers)
Date: 2012
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (4)

Downloads: (external link)
https://econjwatch.org/File+download/588/MayerSept2012.pdf?mimetype=pdf (application/pdf)
https://econjwatch.org/824 (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:ejw:journl:v:9:y:2012:i:3:p:256-297

Access Statistics for this article

Econ Journal Watch is currently edited by Daniel Klein

More articles in Econ Journal Watch from Econ Journal Watch Contact information at EDIRC.
Bibliographic data for series maintained by Jason Briggeman ().