One Swallow Doesn't Make a Summer: A Comment on Zacharias Maniadis, Fabio Tufano, and John List

Mitesh Kataria ()

Econ Journal Watch, 2014, vol. 11, issue 1, 4-10

Abstract: In their article “One Swallow Doesn’t Make a Summer: New Evidence on Anchoring Effects,” Zacharias Maniadis, Fabio Tufano, and John List (2014) present a framework for statistical inference. Presenting evidence from a simulation, the authors suggest that a decision about whether to call an experimental finding noteworthy, or deserving of great attention, should be based on the calculated post-study probability and not on the Classical significance test. In their simulation, it is assumed that the unconditional probability P(H0) is known. The assumption restricts the conclusions to the following: (i) if a non-negligible difference between the post-study probability and the significance level is found then only post-study probabilities offer proper inference; (ii) if a negligible difference is found then both approaches would more or less offer proper inference, though post-study probabilities would be slightly more accurate. In this comment I raise some concerns with their simulation. First, if P(H0) is unknown, as is often the case with economic applications, the post-study probability can lead to even worse inference than the Classical significance test, depending on the quality of the prior. Second, the simulation in Maniadis, Tufano, and List (2014) ignores previous assessments of P(H0) and instead utilizes a selective empirical setup that favors the use of post-study probabilities. Such utilizations might illustrate the pitfalls of the Classical significance test. But, contrary to what Maniadis, Tufano, and List argue, their results do not allow for drawing general recommendations about which approach is the most appropriate.

Keywords: Bayes' theorem; methodology (search for similar items in EconPapers)
JEL-codes: C11 C12 C80 (search for similar items in EconPapers)
Date: 2014
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (2)

Downloads: (external link)
https://econjwatch.org/File+download/792/KatariaJan2014.pdf?mimetype=pdf (application/pdf)
https://econjwatch.org/903 (text/html)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:ejw:journl:v:11:y:2014:i:1:p:4-10

Access Statistics for this article

Econ Journal Watch is currently edited by Daniel Klein

More articles in Econ Journal Watch from Econ Journal Watch Contact information at EDIRC.
Bibliographic data for series maintained by Jason Briggeman ().