Research Commentary ---Too Big to Fail: Large Samples and the p -Value Problem

Mingfeng Lin (), Henry C. Lucas () and Galit Shmueli ()
Additional contact information
Mingfeng Lin: Eller College of Management, University of Arizona, Tucson, Arizona 85721
Henry C. Lucas: Robert Smith School of Business, University of Maryland, College Park, Maryland 20742
Galit Shmueli: Srini Raju Centre for IT & the Networked Economy, Indian School of Business, Hyderabad 500 032, India

Information Systems Research, 2013, vol. 24, issue 4, 906-917

Abstract: The Internet has provided IS researchers with the opportunity to conduct studies with extremely large samples, frequently well over 10,000 observations. There are many advantages to large samples, but researchers using statistical inference must be aware of the p -value problem associated with them. In very large samples, p -values go quickly to zero, and solely relying on p -values can lead the researcher to claim support for results of no practical significance. In a survey of large sample IS research, we found that a significant number of papers rely on a low p -value and the sign of a regression coefficient alone to support their hypotheses. This research commentary recommends a series of actions the researcher can take to mitigate the p -value problem in large samples and illustrates them with an example of over 300,000 camera sales on eBay. We believe that addressing the p -value problem will increase the credibility of large sample IS research as well as provide more insights for readers.

Keywords: empirical modeling; practical significance; effect size; p-value; statistical significance; inference (search for similar items in EconPapers)
Date: 2013
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (93)

Downloads: (external link)
http://dx.doi.org/10.1287/isre.2013.0480 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:orisre:v:24:y:2013:i:4:p:906-917

Access Statistics for this article

More articles in Information Systems Research from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().