Achieving Statistical Significance with Covariates and without Transparency

Gabriel Lenz and Alexander Sahn
Additional contact information
Gabriel Lenz: UC Berkeley

No s42ba, MetaArXiv from Center for Open Science

Abstract: How often do articles depend on suppression effects for their findings? How often do they disclose this fact? By suppression effects, we mean control-variable-induced increases in estimated effect sizes. Researchers generally scrutinize suppression effects as they want reassurance that researchers have a strong explanation for them, especially when the statistical significance of the key finding depends on them. In a re-analysis of observational studies from a leading journal, we find that over 30% of articles depend on suppression effects for statistical significance. Although increases in key effect estimates from including control variables are of course potentially justifiable, none of the articles justify or disclose them. These findings may point to a hole in the review process: journals are accepting articles that depend on suppression effects without readers, reviewers, or editors being made aware.

Date: 2017-04-07
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (4)

Downloads: (external link)
https://osf.io/download/58e82561594d9002510ab37f/

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:osf:metaar:s42ba

DOI: 10.31219/osf.io/s42ba

Access Statistics for this paper

More papers in MetaArXiv from Center for Open Science
Bibliographic data for series maintained by OSF ().