Power boost or source of bias? Monte Carlo evidence on ML covariate adjustment in randomized trials in education

Lukas Fervers
Additional contact information
Lukas Fervers: University of Cologne and Leibniz-Centre for Life-Long Learning

German Stata Conference 2023 from Stata Users Group

Abstract: Statistical theory makes ambiguous predictions about covariate adjustment in randomized trials. While proponents highlight possible efficiency gains, opponents point to possible finite-sample bias, a loss of precision in the case of many and weak covariates, and as the increasing danger of false-positive results due to repeated model specification. This theoretical reasoning suggests that machine learning (variable selection) methods may be promising tools to keep the advantages of covariate adjustment, while simultaneously protecting against its downsides. In this presentation, I rely on recent developments of machine learning methods for causal effects and their implementation in Stata to assess the performance of ML methods in randomized trials. I rely on real-world data and simulate treatment effects on a wide range of different data structures, including different outcomes and sample sizes. (Preliminary) results suggests that ML adjusted estimates are unbiased and show considerable efficiency gains compared with unadjusted analysis. The results are fairly similar between different data structures used and robust to the choice of tuning parameters of the ML estimators. These results tend to support the more optimistic view on covariate adjustment and highlight the potential of ML methods in this field.

Date: 2023-06-15
New Economics Papers: this item is included in nep-big
References: Add references at CitEc
Citations:

Downloads: (external link)
http://repec.org/dsug2023/germany23_Fervers.pdf

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:boc:dsug23:10

Access Statistics for this paper

More papers in German Stata Conference 2023 from Stata Users Group Contact information at EDIRC.
Bibliographic data for series maintained by Christopher F Baum ().