Factorial Designs, Model Selection, and (Incorrect) Inference in Randomized Experiments

Karthik Muralidharan, Mauricio Romero and Kaspar Wüthrich
Additional contact information
Karthik Muralidharan: UC San Diego; NBER; J-PAL
Kaspar Wüthrich: University of Michigan; CESifo

The Review of Economics and Statistics, 2025, vol. 107, issue 3, 589-604

Abstract: Factorial designs are widely used to study multiple treatments in one experiment. Although t-tests using a fully saturated “long model provide valid inferences, “short model t-tests (that ignore interactions) yield higher power if interactions are zero, but incorrect inferences otherwise. Of 27 factorial experiments published in top-five journals (2007–2017), nineteen use the short model. After including interactions, more than half of their results lose significance. Based on recent econometric advances, we show that power improvements over the long model are possible. We provide practical guidance for the design of new experiments and the analysis of completed experiments.

Date: 2025
References: Add references at CitEc
Citations:

Downloads: (external link)
https://doi.org/10.1162/rest_a_01317

Related works:
Working Paper: Factorial Designs, Model Selection, and (Incorrect) Inference in Randomized Experiments (2020)
Working Paper: Factorial Designs, Model Selection, and (Incorrect) Inference in Randomized Experiments (2019)
This item may be available elsewhere in EconPapers: Search for items with the same title.

Export reference: BibTeX RIS (EndNote, ProCite, RefMan) HTML/Text

Persistent link: https://EconPapers.repec.org/RePEc:tpr:restat:v:107:y:2025:i:3:p:589-604

Ordering information: This journal article can be ordered from
https://mitpressjour ... rnal/?issn=0034-6535

Access Statistics for this article

The Review of Economics and Statistics is currently edited by Pierre Azoulay, Olivier Coibion, Will Dobbie, Raymond Fisman, Benjamin R. Handel, Brian A. Jacob, Kareen Rozen, Xiaoxia Shi, Tavneet Suri and Yi Xu

More articles in The Review of Economics and Statistics from MIT Press
Bibliographic data for series maintained by The MIT Press ().