Targeting Prospective Customers: Robustness of Machine-Learning Methods to Typical Data Challenges

Duncan Simester (), Artem Timoshenko () and Spyros I. Zoumpoulis ()
Additional contact information
Duncan Simester: Marketing, MIT Sloan School of Management, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139
Artem Timoshenko: Marketing, MIT Sloan School of Management, Massachusetts Institute of Technology, Cambridge, Massachusetts 02139
Spyros I. Zoumpoulis: Decision Sciences, INSEAD, 77300 Fontainebleau, France

Management Science, 2020, vol. 66, issue 6, 2495-2522

Abstract: We investigate how firms can use the results of field experiments to optimize the targeting of promotions when prospecting for new customers. We evaluate seven widely used machine-learning methods using a series of two large-scale field experiments. The first field experiment generates a common pool of training data for each of the seven methods. We then validate the seven optimized policies provided by each method together with uniform benchmark policies in a second field experiment. The findings not only compare the performance of the targeting methods, but also demonstrate how well the methods address common data challenges. Our results reveal that when the training data are ideal, model-driven methods perform better than distance-driven methods and classification methods. However, the performance advantage vanishes in the presence of challenges that affect the quality of the training data, including the extent to which the training data captures details of the implementation setting. The challenges we study are covariate shift, concept shift, information loss through aggregation, and imbalanced data. Intuitively, the model-driven methods make better use of the information available in the training data, but the performance of these methods is more sensitive to deterioration in the quality of this information. The classification methods we tested performed relatively poorly. We explain the poor performance of the classification methods in our setting and describe how the performance of these methods could be improved.

Keywords: targeting; machine learning; field experiments; covariate shift; concept shift (search for similar items in EconPapers)
Date: 2020
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (16)

Downloads: (external link)
https://doi.org/10.1287/mnsc.2019.3308 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:ormnsc:v:66:y:2020:i:6:p:2495-2522

Access Statistics for this article

More articles in Management Science from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().