Beyond Overall Treatment Effects: Leveraging Covariates in Randomized Experiments Guided by Causal Structure

Ali Tafti () and Galit Shmueli ()
Additional contact information
Ali Tafti: College of Business Administration, University of Illinois at Chicago, Chicago, Illinois 60607
Galit Shmueli: Institute of Service Science, National Tsing Hua University, Hsinchu 30013, Taiwan

Information Systems Research, 2020, vol. 31, issue 4, 1183-1199

Abstract: Researchers using randomized controlled trials (RCTs) often subgroup or condition on auxiliary variables that are not the randomized treatment variable. There are many good reasons to condition on auxiliary variables—also referred to as control variables or covariates—in randomized experiments. In particular, designing and conducting RCTs is costly to researchers and subjects, and therefore it is important to derive greater value from RCT data; measuring not just the average treatment effect (ATE), but also finding more nuanced insights about the underlying theoretical mechanisms and generalizing the inferences. Unfortunately, there are many confusing and even contradictory guidelines on the use of subgroups or auxiliary variables in RCTs. We show how researchers can leverage covariates without biasing their causal inferences, by applying a few simple rules based on Judea Pearl’s causal diagramming framework. We demonstrate how to create a causal schema, through careful and deliberate operationalization of auxiliary covariates, in order to analyze the intermediate effects along a causal chain from the treatment to outcome; and we discuss some other ways to leverage covariates for theory development and generalization of findings from RCTs. We present a criterion for distinguishing pretreatment and posttreatment variables that is based on directed acyclic graphs (DAGs). We provide a succinct set of guidelines to help readers begin to employ some essential techniques of DAG-based causal analysis. Finally, we provide a series of short tutorials (with accompanying simulated data and R scripts) to help readers explore the connections between RCT and observational contexts in causal diagramming. This commentary aims to raise awareness of the DAG methodology, explain its usefulness to experimental research, and encourage adoption in the IS community for studies using RCTs as well as observational data.

Keywords: randomized experiments; causal inference; causal diagram; directed acyclic graph (DAG) (search for similar items in EconPapers)
Date: 2020
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (2)

Downloads: (external link)
https://doi.org/10.1287/isre.2020.0938 (application/pdf)

Related works:
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:inm:orisre:v:31:y:2020:i:4:p:1183-1199

Access Statistics for this article

More articles in Information Systems Research from INFORMS Contact information at EDIRC.
Bibliographic data for series maintained by Chris Asher ().