When Less Is More: Data and Power in Advertising Experiments

Garrett A. Johnson (), Randall A. Lewis () and David H. Reiley ()
Additional contact information
Garrett A. Johnson: Simon Business School, University of Rochester, Rochester, New York 14627
Randall A. Lewis: Netflix, Los Gatos, California 95032
David H. Reiley: Pandora Media, Oakland, California 94612; University of California at Berkeley, Berkeley, California 94720

Marketing Science, 2017, vol. 36, issue 1, 43-53

Abstract: Yahoo! Research partnered with a nationwide retailer to study the effects of online display advertising on both online and in-store purchases. We use a randomized field experiment on 3 million Yahoo! users who are also past customers of the retailer. We find statistically significant evidence that the retailer ads increase sales 3.6% relative to the control group. We show that control ads boost measurement precision by identifying and removing the half of in-campaign sales data that are unaffected by the ads. Less data give us 31% more precision in our estimates—equivalent to increasing our sample to 5.3 million users. By contrast, we only improve precision by 5% when we include additional covariate data to reduce the residual variance in our experimental regression. The covariate-adjustment strategy disappoints despite exceptional consumer-level data including demographics, ad exposure levels, and two years’ worth of past purchase history.

Keywords: advertising effectiveness; field experiments; digital advertising (search for similar items in EconPapers)
Date: 2017
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (11)

Downloads: (external link)
https://doi.org/10.1287/mksc.2016.0998 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:ormksc:v:36:y:2017:i:1:p:43-53

Access Statistics for this article

More articles in Marketing Science from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().