The False Promise of Big Data: Can Data Mining Replace Hypothesis‐Driven Learning in the Identification of Predictive Performance Metrics?

David Apgar

Systems Research and Behavioral Science, 2015, vol. 32, issue 1, 28-49

Abstract: This paper argues US manufacturers still fail to identify metrics that predict performance results despite two decades of intensive investment in data‐mining applications because indicators with the power to predict complex results must have high information content as well as a high impact on those results. But data mining cannot substitute for experimental hypothesis testing in the search for predictive metrics with high information content—not even in the aggregate—because the low‐information metrics it provides require improbably complex theories to explain complex results. So theories should always be simple but predictive factors may need to be complex. This means the widespread belief that data mining can help managers find prescriptions for success is a fantasy. Instead of trying to substitute data mining for experimental hypothesis testing, managers confronted with complex results should lay out specific strategies, test them, adapt them—and repeat the process. Copyright © 2013 John Wiley & Sons, Ltd.

Date: 2015
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
https://doi.org/10.1002/sres.2219

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:bla:srbeha:v:32:y:2015:i:1:p:28-49

Ordering information: This journal article can be ordered from
http://www.blackwell ... bs.asp?ref=1092-7026

Access Statistics for this article

More articles in Systems Research and Behavioral Science from Wiley Blackwell
Bibliographic data for series maintained by Wiley Content Delivery ().