The Sense and Non-Sense of Holdout Sample Validation in the Presence of Endogeneity

Peter Ebbes (), Dominik Papies () and Harald J. van Heerde ()
Additional contact information
Peter Ebbes: Fisher College of Business, Ohio State University, Columbus, Ohio 43210
Dominik Papies: Institute for Marketing and Media, University of Hamburg, 20354 Hamburg, Germany
Harald J. van Heerde: University of Waikato, Hamilton 3240, New Zealand; and Extramural Fellow at CentER, Tilburg University, 5000 LE Tilburg, The Netherlands

Marketing Science, 2011, vol. 30, issue 6, 1115-1122

Abstract: Market response models based on field-generated data need to address potential endogeneity in the regressors to obtain consistent parameter estimates. Another requirement is that market response models predict well in a holdout sample. With both requirements combined, it may seem reasonable to subject an endogeneity-corrected model to a holdout prediction task, and this is quite common in the academic marketing literature. One may be inclined to expect that the consistent parameter estimates obtained via instrumental variables (IV) estimation predict better than the biased ordinary least squares (OLS) estimates. This paper shows that this expectation is incorrect. That is, if the holdout sample is similar to the estimation sample so that the regressors are endogenous in both samples, holdout sample validation favors regression estimates that are not corrected for endogeneity (i.e., OLS) over estimates that are corrected for endogeneity (i.e., IV estimation). We also discuss ways in which holdout samples may be used sensibly in the presence of endogeneity. A key takeaway is that if consistent parameter estimates are the primary model objective, the model should be validated with an exogenous (rather than endogenous) holdout sample.

Keywords: model validation; instrumental variables; IV estimation; endogeneity; exogeneity; predictive model; descriptive model; holdout sample; prediction (search for similar items in EconPapers)
Date: 2011
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (39)

Downloads: (external link)
http://dx.doi.org/10.1287/mksc.1110.0666 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:ormksc:v:30:y:2011:i:6:p:1115-1122

Access Statistics for this article

More articles in Marketing Science from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().