Improving uplift model evaluation on randomized controlled trial data

Björn Bokelmann and Stefan Lessmann

European Journal of Operational Research, 2024, vol. 313, issue 2, 691-707

Abstract: Estimating treatment effects is one of the most challenging and important tasks of data analysts. Personalized medicine, digital marketing, and many other applications demand an efficient allocation of scarce treatments to those individuals who benefit the most. Uplift models support this allocation by estimating how individuals react to a treatment. A major challenge in uplift modeling concerns evaluation. Previous literature suggests methods like the Qini curve and the transformed outcome mean squared error. However, these metrics suffer from variance: their evaluations are strongly affected by random noise in the data, which renders their signals, to a certain degree, arbitrary. We theoretically analyze the variance of uplift evaluation metrics and derive possible methods of variance reduction, which are based on statistical adjustment of the outcome. We derive simple conditions under which the variance reduction methods improve the uplift evaluation metrics and empirically demonstrate their benefits on simulated and real-world data. Our paper provides strong evidence in favor of applying the suggested variance reduction procedures by default when evaluating uplift models on RCT data.

Keywords: OR in marketing; Causal machine learning; Uplift modeling; Evaluation; Qini (search for similar items in EconPapers)
Date: 2024
References: View references in EconPapers View complete reference list from CitEc
Citations:

Downloads: (external link)
http://www.sciencedirect.com/science/article/pii/S037722172300721X
Full text for ScienceDirect subscribers only

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:eee:ejores:v:313:y:2024:i:2:p:691-707

DOI: 10.1016/j.ejor.2023.09.018

Access Statistics for this article

European Journal of Operational Research is currently edited by Roman Slowinski, Jesus Artalejo, Jean-Charles. Billaut, Robert Dyson and Lorenzo Peccati

More articles in European Journal of Operational Research from Elsevier
Bibliographic data for series maintained by Catherine Liu ().