Data pruning in consumer choice models

Elaine Zanutto () and Eric Bradlow ()

Quantitative Marketing and Economics (QME), 2006, vol. 4, issue 3, 267-287

Abstract: Common, if not ubiquitous, Marketing practice when estimating models for scanner panel data is to: (a) observe the data, (b) prune the data to a “manageable” number of brands or SKUs, and (c) fit models to the remaining data. We demonstrate that such pruning practice can lead to significantly different (and potentially biased) elasticities, and hence different managerial/practical outcomes, especially in the context of model misspecification. We first justify our claims theoretically by writing the general problem in a classic missing-data framework and demonstrate that commonly used pruning mechanisms (gleaned from current academic Marketing literature) can lead to a nonignorable missing data mechanism. Secondly, we summarize an extensive set of simulations that were run to understand the driving factors of that bias. The results indicate much greater pruning bias in those cases where model fit is poor (small $$R^{2}$$ ), random utility errors are correlated with the covariates, or the model is misspecified (e.g., a homogeneous logit is specified when a mixed-logit is true). Empirically, we also demonstrate our findings on the well-cited and highly utilized fabric softener data of Fader and Hardie ( 1996 ). Our empirical findings suggest a number of estimates that vary according to the way in which the data is pruned including the magnitude of market mix and attribute elasticities, and purchase probabilities, but that the pruning effect is smaller for better fitting models. Copyright Springer Science + Business Media, LLC 2006

Keywords: Nonignorable missing data mechanism; Probabilistic sampling; Scanner panel data; Multinomial logit models; Mixed-logit models (search for similar items in EconPapers)
Date: 2006
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (13)

Downloads: (external link)
http://hdl.handle.net/10.1007/s11129-005-9000-y (text/html)
Access to full text is restricted to subscribers.

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:kap:qmktec:v:4:y:2006:i:3:p:267-287

Ordering information: This journal article can be ordered from
http://www.springer. ... ng/journal/11129/PS2

DOI: 10.1007/s11129-005-9000-y

Access Statistics for this article

Quantitative Marketing and Economics (QME) is currently edited by Pradeep Chintagunta

More articles in Quantitative Marketing and Economics (QME) from Springer
Bibliographic data for series maintained by Sonal Shukla () and Springer Nature Abstracting and Indexing ().