Latent Stratification for Incrementality Experiments

Ron Berman () and Elea Feit
Additional contact information
Ron Berman: The Wharton School, University of Pennsylvania, Philadelphia, Pennsylvania 19104

Marketing Science, 2024, vol. 43, issue 4, 903-917

Abstract: Incrementality experiments compare customers exposed to a marketing action designed to increase sales with those randomly assigned to a control group. These experiments suffer from noisy responses, which make precise estimation of the average treatment effect (ATE) and marketing return difficult. We develop a model that improves the precision by estimating separate treatment effects for three latent strata defined by potential outcomes in the experiment—customers who would buy regardless of ad exposure, those who would buy only if exposed to ads, and those who would not buy regardless. The overall ATE is estimated by averaging the strata-level effects, and this produces a more precise estimator of the ATE over a wide range of conditions typical of marketing experiments. Analytical results and simulations show that the method decreases the sampling variance of the ATE most when (1) there are large differences in the treatment effect between latent strata and (2) the model used to estimate the strata-level effects is well identified. Applying the procedure to five catalog experiments shows a reduction of 30%–60% in the variance of the overall ATE. This leads to a substantial decrease in decision errors when the estimator is used to determine whether ads should be continued or discontinued.

Keywords: advertising; incrementality experiments; lift testing; A/B testing; holdout experiments; average treatment effect; principal stratification; causal inference (search for similar items in EconPapers)
Date: 2024
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://dx.doi.org/10.1287/mksc.2022.0297 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:ormksc:v:43:y:2024:i:4:p:903-917

Access Statistics for this article

More articles in Marketing Science from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().