Testing What Matters (If You Must Test at All): A Context‐Driven Approach to Substantive and Statistical Significance

Justin H. Gross

American Journal of Political Science, 2015, vol. 59, issue 3, 775-788

Abstract: For over a half century, various fields in the behavioral and social sciences have debated the appropriateness of null hypothesis significance testing (NHST) in the presentation and assessment of research results. A long list of criticisms has fueled the so‐called significance testing controversy. The conventional NHST framework encourages researchers to devote excessive attention to statistical significance while underemphasizing practical (e.g., scientific, substantive, social, political) significance. I introduce a simple, intuitive approach that grounds testing in subject‐area expertise, balancing the dual concerns of detectability and importance. The proposed practical and statistical significance test allows the social scientist to test for real‐world significance, taking into account both sampling error and an assessment of what parameter values should be deemed interesting, given theory. The matter of what constitutes practical significance is left in the hands of the researchers themselves, to be debated as a natural component of inference and interpretation.

Date: 2015
References: Add references at CitEc
Citations: View citations in EconPapers (5)

Downloads: (external link)
https://doi.org/10.1111/ajps.12149

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:wly:amposc:v:59:y:2015:i:3:p:775-788

Access Statistics for this article

More articles in American Journal of Political Science from John Wiley & Sons
Bibliographic data for series maintained by Wiley Content Delivery ().