A Comparison of Approaches to Advertising Measurement: Evidence from Big Field Experiments at Facebook

Brett R. Gordon (), Florian Zettelmeyer (), Neha Bhargava () and Dan Chapsky ()
Additional contact information
Brett R. Gordon: Kellogg School of Management, Northwestern University, Evanston, Illinois 60208
Florian Zettelmeyer: Kellogg School of Management, Northwestern University, Evanston, Illinois 60208; National Bureau of Economic Research, Cambridge, Massachusetts 02138
Neha Bhargava: Facebook Inc., Menlo Park, California 94025
Dan Chapsky: Facebook Inc., Menlo Park, California 94025

Marketing Science, 2019, vol. 38, issue 2, 193-225

Abstract: Measuring the causal effects of digital advertising remains challenging despite the availability of granular data. Unobservable factors make exposure endogenous, and advertising’s effect on outcomes tends to be small. In principle, these concerns could be addressed using randomized controlled trials (RCTs). In practice, few online ad campaigns rely on RCTs and instead use observational methods to estimate ad effects. We assess empirically whether the variation in data typically available in the advertising industry enables observational methods to recover the causal effects of online advertising. Using data from 15 U.S. advertising experiments at Facebook comprising 500 million user-experiment observations and 1.6 billion ad impressions, we contrast the experimental results to those obtained from multiple observational models. The observational methods often fail to produce the same effects as the randomized experiments, even after conditioning on extensive demographic and behavioral variables. In our setting, advances in causal inference methods do not allow us to isolate the exogenous variation needed to estimate the treatment effects. We also characterize the incremental explanatory power our data would require to enable observational methods to successfully measure advertising effects. Our findings suggest that commonly used observational approaches based on the data usually available in the industry often fail to accurately measure the true effect of advertising.

Keywords: digital advertising; field experiments; causal inference; observational methods; advertising measurement (search for similar items in EconPapers)
Date: 2019
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (61)

Downloads: (external link)
https://doi.org/10.1287/mksc.2018.1135 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:ormksc:v:38:y:2019:i:2:p:193-225

Access Statistics for this article

More articles in Marketing Science from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().