RCTs against the Machine: Can Machine Learning Prediction Methods Recover Experimental Treatment Effects?

Brian Prest, Casey Wichman and Karen Palmer

Journal of the Association of Environmental and Resource Economists, 2023, vol. 10, issue 5, 1231 - 1264

Abstract: We investigate how successfully machine-learning (ML) prediction algorithms can be used to estimate causal treatment effects in electricity demand applications with nonexperimental data. We use three prediction algorithms—XGBoost, random forests, and LASSO—to generate counterfactuals using observational data. Using those counterfactuals, we estimate nonexperimental treatment effects and compare them to experimental treatment effects from a randomized experiment for electricity customers who faced critical-peak pricing and information treatments. Our results show that nonexperimental treatment effects based on each algorithm replicate the true treatment effects, even when only using data from treated households. Additionally, when using both treatment households and nonexperimental comparison households, standard two-way fixed effects regressions replicate the experimental benchmark, suggesting little benefit from ML approaches over standard program evaluation methods in that setting.

Date: 2023
References: Add references at CitEc
Citations: View citations in EconPapers (3)

Downloads: (external link)
http://dx.doi.org/10.1086/724518 (application/pdf)
http://dx.doi.org/10.1086/724518 (text/html)
Access to the online full text or PDF requires a subscription.

Related works:
Working Paper: RCTs Against the Machine: Can Machine Learning Prediction Methods Recover Experimental Treatment Effects? (2021)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:ucp:jaerec:doi:10.1086/724518

Access Statistics for this article

More articles in Journal of the Association of Environmental and Resource Economists from University of Chicago Press
Bibliographic data for series maintained by Journals Division ().