RCTs Against the Machine: Can Machine Learning Prediction Methods Recover Experimental Treatment Effects?

Brian Prest, Casey Wichman and Karen Palmer

No 21-30, RFF Working Paper Series from Resources for the Future

Abstract: We investigate how well machine learning counterfactual prediction tools can estimate causal treatment effects. We use three prediction algorithms—XGBoost, random forests, and LASSO—to estimate treatment effects using observational data. We compare those results to causal effects from a randomized experiment for electricity customers who faced critical-peak pricing and information treatments. Our results show that each algorithm replicates the true treatment effects, even when using data from treated households only. Additionally, when using both treatment households and nonexperimental comparison households, simpler difference-in-differences methods replicate the experimental benchmark, suggesting little benefit from ML approaches over standard program evaluation methods.Click "Download" above to read the full paper.

Date: 2021-09-29
New Economics Papers: this item is included in nep-big, nep-cmp, nep-exp and nep-inv
References: Add references at CitEc
Citations:

Downloads: (external link)
https://www.rff.org/documents/3094/RCTs_Against_the_Machine.pdf (application/pdf)

Related works:
Journal Article: RCTs against the Machine: Can Machine Learning Prediction Methods Recover Experimental Treatment Effects? (2023)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:rff:dpaper:dp-21-30

Access Statistics for this paper

More papers in RFF Working Paper Series from Resources for the Future Contact information at EDIRC.
Bibliographic data for series maintained by Resources for the Future ().