Factorial Designs, Model Selection, and (Incorrect) Inference in Randomized Experiments

Karthik Muralidharan, Mauricio Romero and Kaspar Wüthrich

No 8137, CESifo Working Paper Series from CESifo

Abstract: Factorial designs are widely used for studying multiple treatments in one experiment. While “long” model t-tests provide valid inferences, “short” model t-tests (ignoring interactions) yield higher power if interactions are zero, but incorrect inferences otherwise. Of 27 factorial experiments published in top-5 journals (2007–2017), 19 use the short model. After including all interactions, over half their results lose significance. Modest local power improvements over the long model are possible, but with lower power for most values of the interaction. If interactions are not of interest, leaving the interaction cells empty yields valid inferences and global power improvements.

Keywords: randomized controlled trial; factorial designs; cross-cut designs; field experiments (search for similar items in EconPapers)
JEL-codes: C12 C18 C90 C93 (search for similar items in EconPapers)
Date: 2020
New Economics Papers: this item is included in nep-exp
References: View references in EconPapers View complete reference list from CitEc
Citations: View citations in EconPapers (12)

Downloads: (external link)
https://www.cesifo.org/DocDL/cesifo1_wp8137.pdf (application/pdf)

Related works:
Working Paper: Factorial Designs, Model Selection, and (Incorrect) Inference in Randomized Experiments (2019)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:ces:ceswps:_8137

Access Statistics for this paper

More papers in CESifo Working Paper Series from CESifo Contact information at EDIRC.
Bibliographic data for series maintained by Klaus Wohlrabe ().